1. ess_es_server.jar
2. ess_japi.jar
3. odihapp_common.jar
4. odihapp_essbase.jar

Out of these 4, the first 2 are essbase jdbc connectivity jar files. The remaining 2 contain the core code for reverse engineering the Essbase Metadata (RKM). This is not a straightforward JDBC interface. So, how do we leverage this within the OWB framework. This is where David Allan was extremely kind enough to get me started. To reverse engineer any metadata in OWB, through an external API requires 3 main tasks

1. An external Stub Driver which should be accessible to the OWB code path – This is the base for enabling custom API connectivity – The driver is available here for download

2. Platform Definition – Since Essbase does not exist as a platform in OWB, we need to manually define them using a set of OMB Plus scripts. There is a platform definition XML file which can be download from here.

3. Necessary JDBC API jar files in OWB Code path

In the first step, we have to basically copy the Stub jar file to the {ORACLE_HOME}/owb/lib/ext directory.

Once the stud driver is copied over, we need to copy the Platform definition XML file to the {ORACLE_HOME}/owb/bin/admin directory. This file will be used from our OMB Plus scripts while defining the platform.

To define the platform, run the following commands from within OMBPlus. Ensure that your working directory is {ORACLE_HOME}/owb/bin

OMBCONNECT OWB_WS_OWNER/welcome1@ORCL USE WORKSPACE 'OWB_WS_OWNER.WORKSPACE1'
OMBCREATE PLATFORM 'HYPERION_ESSBASE' SET PROPERTIES (BUSINESS_NAME) VALUES ('Hyperion Essbase')
OMBALTER PLATFORM 'HYPERION_ESSBASE' SET PROPERTIES (DRIVER_CLASS,URI_TEMPLATE) VALUES ('oracle.owb.jdbc.StubDriver','false:Accounts:Accounts:false')
OMBALTER PLATFORM 'HYPERION_ESSBASE' SET PROPERTIES (LOCAL_OBJECT_MASK) VALUES ('%SCHEMA.%OBJECT')
OMBALTER PLATFORM 'HYPERION_ESSBASE' SET  PROPERTIES (DDLNULL) VALUES ('null')
OMBALTER PLATFORM 'HYPERION_ESSBASE' SET PROPERTIES (LOCAL_OBJECT_MASK) VALUES ('%OBJECT')
OMBALTER PLATFORM 'HYPERION_ESSBASE' SET PROPERTIES (DEFAULT_MAX_NAME_LEN) VALUES ('35')
OMBALTER PLATFORM 'HYPERION_ESSBASE' SET PROPERTIES (REMOTE_OBJECT_MASK) VALUES ('%OBJECT')

OMBALTER PLATFORM 'HYPERION_ESSBASE' ADD PLATFORM_TYPE 'NUMERIC'
OMBALTER PLATFORM 'HYPERION_ESSBASE' MODIFY PLATFORM_TYPE 'NUMERIC' SET PROPERTIES(SYNTAX) VALUES ('new Integer(%precision,%scale)')
OMBALTER PLATFORM 'HYPERION_ESSBASE' MODIFY PLATFORM_TYPE 'NUMERIC' SET PROPERTIES (P1,P1MAX,P1DEFAULT,P1TYPE) VALUES ('precision','3000', '0','range')
OMBALTER PLATFORM 'HYPERION_ESSBASE' MODIFY PLATFORM_TYPE 'NUMERIC' SET PROPERTIES (P2, P2MIN, P2MAX, P2TYPE) VALUES ('scale', '0', '18', 'range')

OMBALTER PLATFORM 'HYPERION_ESSBASE' ADD FROM_PLATFORM_TYPEMAP 'NUMERIC_TOG' SET PROPERTIES (FROM_DATATYPE, TO_DATATYPE) VALUES ('NUMERIC', 'NUMERIC')
OMBALTER PLATFORM 'HYPERION_ESSBASE' ADD TO_PLATFORM_TYPEMAP 'NUMERIC_FROMG' SET PROPERTIES (FROM_DATATYPE, TO_DATATYPE) VALUES ('NUMERIC', 'NUMERIC')

OMBALTER PLATFORM 'HYPERION_ESSBASE' ADD PLATFORM_TYPE 'STRING'
OMBALTER PLATFORM 'HYPERION_ESSBASE' MODIFY PLATFORM_TYPE 'STRING' SET PROPERTIES(SYNTAX) VALUES ('new String(%size)')
OMBALTER PLATFORM 'HYPERION_ESSBASE' MODIFY PLATFORM_TYPE 'STRING' SET PROPERTIES (P1,P1MAX,P1DEFAULT,P1TYPE) VALUES ('size','3000', '1','range')
OMBALTER PLATFORM 'HYPERION_ESSBASE' ADD FROM_PLATFORM_TYPEMAP 'STRING_TOG' SET PROPERTIES (FROM_DATATYPE, TO_DATATYPE) VALUES ('STRING', 'VARCHAR')
OMBALTER PLATFORM 'HYPERION_ESSBASE' ADD TO_PLATFORM_TYPEMAP 'STRING_FROMG' SET PROPERTIES (FROM_DATATYPE, TO_DATATYPE) VALUES ('VARCHAR', 'STRING')

OMBALTER PLATFORM 'HYPERION_ESSBASE' ADD PLATFORM_TYPE 'DATE'
OMBALTER PLATFORM 'HYPERION_ESSBASE' MODIFY PLATFORM_TYPE 'DATE' SET PROPERTIES(SYNTAX) VALUES ('new Date()')
OMBALTER PLATFORM 'HYPERION_ESSBASE' ADD FROM_PLATFORM_TYPEMAP 'DATE_TOG' SET PROPERTIES (FROM_DATATYPE, TO_DATATYPE) VALUES ('DATE', 'DATE')

When you connect using OMBCONNECT, ensure that you are connecting to your workspace using the Workspace owner username/password. As you see, this basically defines certain extrinsic and intrinsic properties of the new platform that we have defined. Thanks to David Allan for providing the necessary stub driver, platform definition XML files. Without these there is no way i would have got the connectivity to work.

Once the platform is successfully defined, you should see the new platform within OWB as shown below.

So far so good. The next step is to copy the 4 ODI jar files (ess_japi.jar, ess_es_server.jar, odihapp_common.jar and odihapp_essbase.jar) from {ODI_HOME}/drivers/ directory to {ORACLE_HOME}/owb/lib/ext directory. I could not get this to work with 10.1.3.5 version of these jar files. Somehow it was always using a 9.3 version of the provider services for connecting to essbase. So, i had to upgrade to 10.1.3.6 of ODI to get this working. But David did mention the fact that it will work 10.1.3.5 version as well. Remember, just to use the jar files itself requires the license of ODI. Also, check with Oracle support on whether this is supported. Though there is no reason why this will not work, since it is not documented, i doubt Oracle Support would actually support this.

Restart the OWB client. In the new “Hyperion Essbase” platform, create a new module as shown below

In the Locations screen, enter the connection details to Essbase as shown below

The schema will be the Essbase Database name. In my case, i am connecting to the Sample->Basic cube. So Basic is the schema name. The port will be the essbase port (1423 is the default). The username/password will be the one that you will be using for connecting to Essbase. The Driver Class will be automatically populated as we had copied the Stub Driver class already. But the URL needs to be populated. the URL basically points to the options that we get while importing Essbase metadata through ODI RKM’s. In ODI, while choosing an RKM, we need to basically choose whether we want to import the measure dimension as a flat list or as a separate column. Since we are using the same RKM, the parameters will be passed through the URL as

<MultipleDataColumns>:<Account Dimension>:<AccountDimensionMembers>:<AttribMem>

In my case, i am importing the Accounts dimension as a separate column. So, my URL will look like

false:Measures:Accounts:false

In the Advanced tab, enter the Application ,database and the server names as shown below.

Once this is done, import the metadata tables in the same fashion as we do on a normal database connection. The screenshots below should give you an idea of how the Essbase metadata will be imported

As you see, OWB is now able to import Essbase metadata using the ODI RKMs. This is extremely good and shows the extensibility of OWB which to me has been the biggest downside in the prior releases. Effectively one can use this to import custom application metadata using any JDBC API. The SAP, PSFT connectors in OWB basically extend this concept by connecting through an API instead of directly to the database. The next in line is how can we use this imported Cube in a ETL process in OWB using OWB Code Templates.

As a principle, I hate the idea of having to update a fact. A fact has happened, it is not going to change. I suppose to be more accurate a “change” is a new event, a new fact, a new fact occurring at a different time. So how to model this? – well instinctively I would go for a table that is only inserted (preferably appended to – think set based!) containing whatever dimensions are needed (don’t forget the ‘when’ dimension) PLUS an ‘EVENT’ dimension (one row per expected status) and the measures (how many, how much etc). To report on this we need to rotate the table so that the events that belong to single item appear in the same row. Before Oracle 11g we would need to construct some SQL using a mix of case statements and analytic functions to rotate the data. But now we have a potentially better way the, Oracle 11g Pivot operator.

Here we define a set of dimensions for the row (similar to the dimensions in a Group BY clause), the aggregation operators for the pivoted measures – which of course could include MIN() or MAX() for the cases when want to pivot DATE types. We also need to define the dimensions we want to pivot by, and here we can actually choose multiple dimensions; this again is somewhat similar to the GROUP BY of traditional SQL. Remember though when we pivot we sometimes only expect to ‘aggregate’ a single row – if we want to pivot order date and dispatch date then we probably have just one of each!.

So how does it look? Well the Oracle 11g documentation describes the syntax and gives some examples – here I am showing a slightly more complicated case where we are pivoting by two dimensions, each with a known set of code values. This example is based on two of the examples in the Oracle 11g Data Warehousing Guide

	SELECT * FROM	(
		SELECT product, channel, quarter, quantity_sold FROM sales_view
		) PIVOT (SUM(quantity_sold) as SUMQ, SUM(amount_sold) as SUMS
			FOR (channel, quarter) IN
			((5, '02') AS CATALOG_Q2,
		 	(4, '01') AS INTERNET_Q1,
		 	(4, '04') AS INTERNET_Q4,
		 	(2, '02') AS PARTNERS_Q2,
		 	(9, '03') AS TELE_Q3
			) );

The query returns a column for the product and for each of the specified pairs of channel and quarter a column for each measure. So we get columns for:

PRODUCT, CATALOG_Q2_SUMQ, CATALOG_Q2_SUMS, INTERNET_Q1_SUMQ, INTERNET_Q1_SUMS, INTERNET_Q4_SUMQ, INTERNET_Q4_SUMS, PARTNERS_Q2_SUMQ, INTERNET_Q4_SUMS, TELE_Q3_SUMQ, and TELE_Q3_SUMS

Note how the the measure name is concatenated to the alias in the in list.
As you can see we don’t need to specify each combination of channel and quarter – just the ones we want in our pivoted view. We also don’t use a GROUP BY clause – we specify the columns we want to see (both the dimensions and the aggregations) and Oracle implicitly groups by all of the columns not in aggregated functions.

In my example I used SELECT * to wrap the inline pivot, in practice I would explicitly select the columns and perhaps alias them to more meaningful names than the concatenated ones generated by Oracle. I would also expose the pivot as database view and thus access it from OWB or OBIEE where it appears to be just another table or view.

Another point to note is that you might see null values in the pivoted measures and these can be due to one of two reasons: the value stored for that combination of dimensions (in our case channel and quarter) is actually NULL, or that the combination does not exist. If you need to (and you may not need to) you can differentiate by using a COUNT measure; if the count is zero then the combination does not exist in the source table, if one or more then the source has NULLs stored for the combination.

We used a similar pivot view to the one above to monitor stock movements in a warehouse – in this case we needed to track individual batches of product from multiple potential suppliers, so in addition to the product dimension we had dimensional columns for batch id (a degenerate dimension) and supplier. The view was then exposed to OWB to allow us to include the aggregated result set in our ETL process – we needed to calculate some additional measures based on the difference between two of the pivoted columns. The Pivot operator greatly simplified our ETL for this fact – we could easily write an ETL process with a straight aggregation then pivot the results with CASE statements or DECODES or whatever – but that would have been less clear and also increased the number of “moving parts”.

We have had no problems with performance with our data set – 80 million rows pivoted on Exadata to just a few seconds. But it was not too slow on our non-exadata development machine either.

With involvement from diverse groups, I wanted to make sure the presentation would be interesting from a lot of perspectives, so involving core database functionality like OWB seemed important. And this particular presentation is directed at DBAs and developers, so the non-BI representation from GOUSER might be interested in what’s going on.

If you’re new to OWB, don’t be alarmed by the focus on the newest release: 11gR2. I’ll explain in general what OWB is and how it’s positioned, moving into some of the new features that make this release the best OWB yet. Also speaking at the event will be Mitch Campbell, Director of Business Intelligence with Oracle, who will be talking about Oracle’s BI product roadmap. Following both talks, I’ll be sitting for a BI panel discussion with Mitch and Greg, to give responses to attendees questions, hopefully representing the perspectives of Oracle, their partners, and their customers.

The event is free, with snacks and refreshments provided, so hopefully we’ll see you there.

To create an example, we have a table called CUSTOMERS_TGT that has a column called GENDER. We want to enforce a data rule that says this column cannot be null. However we want our Warehouse Builder mapping to catch rows coming in that have null in this column, and replace the nulls with an “Unknown” value. To start this process we open up the target table in the Data Object Editor (I’m using version 11gR1 of Warehouse Builder, but the feature works the same in 10gR2 and 11gR2), click on the Data Rules tab at the bottom, and click on the Apply Rule button.

I am then presented with the first page of a wizard, and then I’m shown a list of pre-defined data quality checks that I can select from. I select the NOT NULL check and press Next to continue, I then name the data quality check and are then asked which table column the check should be “bound” to. I select the GENDER column and press Next to continue.

Once the wizard is complete, my table now has an entry under Data Rules, with this Not Null rule shown as applying and bound to the GENDER column.

I then use the Control Center Manager to redeploy this table, not because the new data rule has caused a physical NOT NULL constraint to be added to the target table (all of these constraints are “virtual” and handled within the Warehouse Builder mapping), but because of a “Shadow Table” that needs to be deployed alongside the target table to handle the errors. You don’t see this table listed in the set of tables within the OWB database module, it only appears as a script when you go to redeploy the table, but it’s important you deploy this (or create your own of the same definition) otherwise the mappings later on will fail.

Note also that your target table shouldn’t have a real NOT NULL constraint on it , as this can cause your mapping to fail even when error handling is enabled in the mapping, as the real constrain gets in there first before your mapping can gracefully handle the error. The target table should however have a primary key as this is used later on when moving incoming rows around that fail the error check.

Over to the mapping editor now, where I’ve got a simple mapping that loads from a source table into this target table. Note that my target table, the one above, has some extra rows in the table operator now due to the data rule that I’ve added to it.

As you can see, the target table has some extra rows that are used to hold any incoming rows that fail data rule checks, with the ERR$$$ columns holding the reason for the error. Before the error handling will take place though you need to tell OWB, via the target table properties editor, to move the errors to the error handling table rather than just ignore, or report on them.

Note also the Error Table section of the table properties, which lets you substitute your own, pre-built error handling table if you want to, and to fine-tune how error records are stored in the table.

So now we’re ready to go. I deploy the mapping (if the mapping deployment fails because of a table it can’t find, check that you’ve deployed the error (shadow) table mentioned above) and then run it. Even though my incoming data had one row with a null value in the GENDER column, the mapping completes successfully.

However the Job Details page shows that some incoming data was diverted to my error handling table.

SQL Developer shows that the new error handling table does in fact contain the row that’s failed the data quality check.

Checking the target table shows that all of the remaining rows are compliant with the NOT NULL data rule.

But what if we wanted to correct data that failed the check, as well as record it in this error handling table. Well, we can take the output of the error rows in the target table operator, apply a transformation to it and then load just these back into the target table, if there’s a “not known” value for example that we wish to apply to values that would otherwise be null.

Make sure you set the error handling action of this new target table operator to also reject rows that fail the not null check (otherwise you might inadvertently add the row back with data that might still fail the data rule check).

Running the mapping again shows it executing correctly, but this time if I check SQL Developer, I can see a new row appearing that has had its otherwise null GENDER value converted to a “U”, for unknown.

The mapping execution still added a row to the error table though for this row, so that I can see where the mapping has corrected data in the background.

The process works in the same way for other data rules you might apply, such as domain check rules, foreign key rules, “not number” rules and so forth. One thing to bear in mind though is that these mappings run in PL/SQL (as opposed to SQL, set-based) mode and therefore might not perform as well as ones without data rules applied [This was fixed from 10.2.0.3, see the note from David Allen in the comments below, these mappings now run set-based in the same way as other mappings]. However if you think about the benefits in terms of graceful error handling, the “self-documenting” nature of mappings that use this feature and so on, it’s certainly a good way of handling data quality issues in your staging or integration layer before bulk-loading the data into your main warehouse tables.

So, I set out to give it a go: Can I run the Linux installed version of OWB11gR2 on Windows? And the answer is simple: It’s easy as pie

First you need to download the stand-alone version linux version of the binaries. Follow the instruction and install the software on a linux box. I used an Oracle REL VM for this purpose. Once the installer has completed we need to copy the whole OWB home over to our Windows box (I used Windows XP VM). Once we get the 1GB directory copied over there are a few minor steps we must take:

1. Make sure we have a java runtime to use. The Linux install will not include a Windows version of JRE/JDK. I had an Oracle DB installed on my VM so I used the JDK that came with that installation.
2. Create a valid setowbenv.bat file that takes care of setting up the OWB runtime environment. The $OWB_HOME/bin/win32/ directory contains a template file, setownenv.tmp for this, so go ahead and make a working version.
3. Run the owbclient.bat file from the £OWB_HOME/bin/win32/ directory and you are good to go

Since you need to connect to an OWB11gR2 repository, I simply followed the documents to run against an 11gR1 database on my VM. To put it simple you need to upgrade the pre-sseded OWBSYS schema that comes with all 11g Oracle instances to the R2 version. After that you are good to go …

UPDATE: To sum up the extra steps I have put in the comments here:

1. To get the Control Center Agent running (the bit that takes care of the new Code Template technology): Edit the %OWBHOME%\owb\jrt\config\secure-web-site.xml file and fix the absolute path of the ssl-config keystore entry
2. To get the Control Center Service running (the bit that takes (and always has taken) care of the regular mappings): Make sure that the %JAVAHOME% variable is set to a proper Java home, in the %OWBHOME%\owb\bin\win32\run_service_local.bat file. Then start the service using the local_service_login.bat script.

/

]]> http://www.rittmanmead.com/2009/12/10/owb11gr2-on-windows/feed/ 16 http://www.rittmanmead.com/2009/11/30/ukoug-tech-and-ebs-conference-day-1/ http://www.rittmanmead.com/2009/11/30/ukoug-tech-and-ebs-conference-day-1/#comments Mon, 30 Nov 2009 22:38:53 +0000 Mark Rittman http://www.rittmanmead.com/?p=3704 I’m now back in my hotel room, having escaped from the Focus Pubs at the end of Day 1 of the UKOUG Technology and E-Business Suite Conference in Birmingham. It’s been a long day, with an 8am meeting on the stand in the exhibition, a stint in the speaker ready room prepping my demonstrations, and then delivery of two sessions this afternoon on OBIEE, Essbase and Oracle Warehouse Builder 11gR2.

This year we’ve brought most of the consulting team along to the conference, as a way of getting to see people who are often working away on-site, and so that everyone can get to meet the great delegates, speakers and partners at the conference. Here’s the team at the start of the day, having just set up the demo and waiting for the opening of the exhibition.



And here’s things at the end of the day, during the Exhibition Hall Drinks Reception. We had our famous Rittman Mead beer available again, and it was great to catch up with people such as Paul Vallee, Connor McDonald, Jonathan Lewis, Lisa Dobson and everyone else who stopped by.



The two sessions we ran today were on Essbase and OBIEE Integration, and on the new 11gR2 release of OWB. The Essbase and OBIEE session went well; we had Venkat doing the demos, myself doing the positioning, and then a great questions and answer session at the end where we discussed attendees experiences with the integration and gave some feedback on what worked for us. The slides are available here if you’re interested.

The second session was particularly well attended and was on the 11gR2 version of OWB. Now if you’ve been following our blog, we’ve posted a series of articles on this new release, and we’re particularly excited by the new code template (ODI) features in this release, especially as the integration between the ODI and OWB technologies seems to have been done so well. We were lucky enough to be joined by Craig Stewart, our Oracle ODI representative in EMEA and someone we know well from his attendance at various user group events and indeed our own BI Forum earlier in the year in Brighton. Craig was able provide a “hot off the presses” roadmap slide on OWB and ODI 12g which he’s kindly agreed for me to reproduce here:



What we can take from this are a few points:

• Going forward, the OWB product name will be de-emphasized in favour of “In-Database ETL”, to differentiate itself from Oracle’s strategic data integration platform, ODI
• The 11gR2 version of OWB will be the last in it’s current packaging, and license-wise it’ll now be incorporated into ODI Enterprise Edition. The free OWB functionality will still remain but will be renamed “In-Database ETL” as above.
• ODI 12g is the target version for integration between the ODI and OWB platforms
• There will be an ODI 11g release out in the first half of next year (usual disclaimers apply
• ODI is the strategic data integration platform for Oracle

Again, the slides are available for download here if you want to take a look, and they cover code templates, heterogeneous sources, OBIEE integration, change data capture and web service integration.

Craig is actually going to join me again tomorrow for the ODI and Oracle BI Applications session I’ll be running (14.30 – 15.30, Tuesday, Hall 10b), so again if you’ve got questions on the ETL direction for the BI Applications and the role that ODI will play, come along and we’ll be pleased to answer any questions.

Finally, if you’re around tomorrow (Tuesday) evening, we’re running a BI “Appreciation” drinks session at the Pitcher and Piano, Brindley Place, from 6pm – 9pm. If you’re a BI user, customer, developer, partner etc, or you just know us and would like to join us for drinks and canapes, come down in the evening and it’ll be great to see you.

]]> http://www.rittmanmead.com/2009/11/30/ukoug-tech-and-ebs-conference-day-1/feed/ 2 http://www.rittmanmead.com/2009/10/31/owb-11gr2-workspace-creation-issues/ http://www.rittmanmead.com/2009/10/31/owb-11gr2-workspace-creation-issues/#comments Sat, 31 Oct 2009 20:41:03 +0000 Stewart Bryson http://www.rittmanmead.com/?p=3502 First of all, I’d like to thank David Allan of the OWB Product Team for helping me solve the problem I discuss below… and also for identifying what he thinks to be the cause.

A few days after Oracle Database 11gR2 was available, I went about building my first VM to test both the database features as well as OWB. I installed 64-bit Oracle Enterprise Linux 5.4 on a VM and then installed the 64-bit version of Oracle Database 11gR2. The database install went just fine, and once complete I executed $ORACLE_HOME/owb/bin/owb to get OWB started, specifying a valid username and password. OWB recognizes that my username is not a registered user of an existing workspace, so it asks me if I want to create a new workspace… or at least register with an existing one.



I select yes, which immediately launches the Repository Assistant. I select the first option, “Manage Warehouse Builder Workspaces”, and then again select the first option: “Create a new Warehouse Builder workspace”. At the next screen, I select “Create a workspace with a new user as workspace owner”.



At Step 6, I provide a username for my new workspace, a password, and a name for the workspace. Then I select Next.



At Step 10, I go ahead and select the username STEWART to be added as a workspace user, and then select next. I then get a summary screen, where I press Finish. Immediately, I get an error involving the installation of the OWBSYS user. The actual exception is “oracle.ide.ExitNotAllowedException” in the process called “processLoadJavaToken”.



I look at the log generated by the Repository Assistant in the $ORACLE_HOME/owb/UnifiedRepos directory… but the only additional piece of information supplied is the following: “Error occurred during Seeding OWBSYS”. This was all quite confusing, as I knew that Mark had successfully built OWB with no issues, and so had Pete Scott, Venkat and others.

At this point, I contacted David Allan and asked if he had seen this error, which he hadn’t. He looked into it a bit further, but in the end, he simply recommended that I clean out the OWBSYS user and then try to “seed” it manually, which simply means creating the required objects in the schema for supporting new workspaces. This is of course what the Repository Assistant is trying to do, but for whatever reason, the seed process in the GUI is not working correctly for me.

To clean out the OWBSYS schema, I use the clean_owbsys.sql script in the $ORACLE_HOME/owb/UnifiedRepos directory. This will drop the OWBSYS user and all roles associated with it.

SQL> @clean_owbsys

User dropped.

User dropped.

Role dropped.

Role dropped.

Role dropped.

SQL>

I follow that up with cat_owb.sql script, which recreates OWBSYS and seeds all the required objects. I truncated some of the output for brevity sake, which I indicated with the “…” characters.

SQL> @cat_owb
Enter Tablespace Name for OWBSYS user:
USERS

Package created.

Package body created.

Create user OWBSYS with default tablespace USERS ...
old   1: CREATE USER OWBSYS IDENTIFIED BY OWBSYS DEFAULT TABLESPACE &owbsys_tbs
new   1: CREATE USER OWBSYS IDENTIFIED BY OWBSYS DEFAULT TABLESPACE USERS

User created.

User OWBSYS has been created.

PL/SQL procedure successfully completed.

Grant succeeded.

...

Grant succeeded.

PL/SQL procedure successfully completed.

PL/SQL procedure successfully completed.

Commit complete.

...

PL/SQL procedure successfully completed.

Commit complete.

Role created.

Grant succeeded.

Role created.

Grant succeeded.

Role created.

Grant succeeded.

...

Grant succeeded.

old   1: CREATE USER owbsys_audit IDENTIFIED BY owbsys_audit DEFAULT TABLESPACE &owbsys_tbs
new   1: CREATE USER owbsys_audit IDENTIFIED BY owbsys_audit DEFAULT TABLESPACE USERS

User created.

Grant succeeded.

Grant succeeded.

Synonym created.

...

Synonym created.

Table created.

PL/SQL procedure successfully completed.

User altered.

User altered.

NOTE: To associate the Control Center with the correct OWB home, now run owb/UnifiedRepos/reset_owbcc_home.sql. You must run the script as a user with java admin privileges. For example, connect as sys as sysdba to run the script.
SQL>

I follow the advice from the cat_owb.sql script, and I execute the reset_owbcc_home.sql.

SQL> @reset_owbcc_home
Enter the full path of the Oracle home for the OWB Control Center install.
If you are installing in a Windows environment, please ensure that the case of the path exactly matches the Oracle install path including the drive letter.

OWB Control Center Home:
/oracle/product/11.2.0/dbhome_1

PL/SQL procedure successfully completed.

PL/SQL procedure successfully completed.

SQL>

The cat_owb.sql script recreates and locks the OWBSYS account… so I unlock the account and change the password. Now, when I run the repository assistant I get a different result.



David Allan’s assessment of the cause of this issue is the use of the client tools, namely the Design Client and the Repository Assistant, on a 64-bit Linux platform. According to the certification matrix, none of the OWB client tools are certified on 64-bit Linux. That’s very strange, as it’s unclear why any of the client tools would continue to ship on the 64-bit installations if they shouldn’t be used.

Remember, both the Oracle Database and OWB continue to be supported on 64-bit Linux, it’s just my use of the client tools from that same installation that is not supported. Were I to download a 32-bit Linux installation and use the client tools from that installation to connect to a 64-bit installation… I’d be using a certified solution. And when Oracle starts shipping 11gR2 on Windows, the use of the 32-bit Windows client installation to connect to the 64-bit database installation is supported as well.

I can certainly work around the workspace creation issue, as I showed above, by doing the OWBSYS seeding manually. However, as David points out, I’m opening myself up for lots of issues by using a non-certified OWB client.

]]> http://www.rittmanmead.com/2009/10/31/owb-11gr2-workspace-creation-issues/feed/ 8 http://www.rittmanmead.com/2009/10/29/upgrading-an-owb-project-to-11gr2/ http://www.rittmanmead.com/2009/10/29/upgrading-an-owb-project-to-11gr2/#comments Thu, 29 Oct 2009 15:50:41 +0000 Stewart Bryson http://www.rittmanmead.com/?p=3481 Mark has been out front and center in demonstrating the new features of OWB 11gR2, both in the OWB Unconference and in numerous blog postings over the last few months. But one thing we haven’t covered yet on the blog is what the upgrade of a current OWB project looks like. I will be demonstrating how to upgrade an 11.1.0.6 OWB project to 11gR2, but I will also point out some of the different steps that should be taken if your version of OWB is 10g. The database I’m going to be upgrading is from the VM I put together to do this blog posting… it consists of two mappings and a single process flow.

As OWB installs as part of the database now, the upgrade to 11gR2 begins with an update to the database, and regardless of the intial platform, most of the database upgrade looks the same. I start by running the database installer, and when prompted for configurations preferences, I select “Upgrade an existing database”.



After the software install is complete, the Database Upgrade Assistant (DBUA) launches automatically to handle upgrading existing databases. The Assistant handles the basic tasks of starting the database with STARTUP UPGRADE, and then executing the appropriate catalog scripts to manage the modifications to the data dictionary, as well as other stored objects owned by the various default Oracle schemas.



During the database upgrade, I can see all the different components of the Oracle Server being upgraded, including the Java server, the XML server, the OLAP catalog, etc.



The only interesting thing about my database upgrade is that the process changed the default port for the listener from 1521 to 1522. When I create demonstration database VM’s on my Mac, I typically use dynamic listener registration, which eliminates the need for a listener.ora file. Instead, the database uses the information in the pfile or spfile to register services to listen for, but I’m not really sure why the Upgrade Assistant thought I needed to change the port number.

This change is problematic for OWB installations, as I already know that I will have to modify all the database and process flow locations to use this new port number. It’s a huge inconvenience that I could eliminate by moving away from dynamic listener registration and creating a listener.ora file. However, it seems like a decent thing to work through in the name of science, so I leave it and move on.

After the database upgrade is complete, I now have the database mounted with the new ORACLE_HOME, therefore the 11gR2 binaries and database objects are available for this database, and if I were to create a new OWB project, then that project would use the 11gR2 version of OWB. But pre-existing OWB projects don’t get upgraded automatically, and there’s quite bit of massaging required to make this happen. The process of upgrading the OWB repository is one instance where the upgrade steps will differ depending on which database version is being upgraded. Because 11gR1 already has the concept of the “workspace” built in, the schema OWBSYS already exists, so the main modification to the repository is the addition of the OWBSYS_AUDIT schema. To make the required change to the repository, the $ORACLE_HOME/owb/UnifiedRepos/cat_owb_reposupgrade.sql script is executed as SYSDBA.

SQL> @cat_owb_reposupgrade

Role altered.

Grant succeeded.

Grant succeeded.

PL/SQL procedure successfully completed.

PL/SQL procedure successfully completed.

Commit complete.

If the upgrade is performed against a 10gR2 database, the OWBSYS schema doesn’t exist yet, and needs to be created. In this case, the $ORACLE_HOME/owb/UnifiedRepos/cat_owb.sql should be executed instead.

The next step is to run the OWB Repository Assistant to complete the migration of the repository. I’m not exactly sure why the Repository Assistant couldn’t have included the previous step of running either the cat_owb_reposupgrade.sql or cat_owb.sql scripts. It’s not a major effort to do that step manually, but simplifying the process wherever possible is a definite benefit.

On Linux, the Repository Assistant is executed using the $ORACLE_HOME/owb/bin/unix/reposinst.sh script. One of the first questions I get asked is what sort of operation I would like to complete, and I select “Upgrade repository to current release of Oracle Warehouse Builder”.



I am then asked what sort of upgrade option I would like to perform. I choose “Upgrade repository on the same database instance”, and then the upgrade process takes off.



The other options include “Export entire repository to a file” or “Import entire repository from a file”. According to the documentation, we would have to perform these two steps manually if our repository was 10g, first exporting the entire contents to a file, and then importing them. I don’t know what aspect of the 10g upgrade limits the “upgrade in place” option. Conceptually, the lack of workspace functionality is quite a significant change, but I’m not certain it’s significant enough that it can’t be handled by a single migration option in the Repository Assistant.

Anyway.. back to 11gR1. While the Repository Upgrade is underway, we see several different informational messages concerning the status of the upgrade. Finally, I get a message that states: “Upgrading repository completed successfully”. Not the best grammar… but it’ll do.



Once the migration is complete, I launch the OWB Client to have a look around and notice that the mappings and process flows are all deployed, and the deployment and execution history is all intact. However, I know I have to modify the port number for all the locations (because it was changed to 1522 during the database upgrade), and also update the version from 11.1 to 11.2 for all the database locations. There are two options for completing this, and because I’m using a VM, I was able to test both options by first trying one, rolling back, and then trying the other. Both worked successfully for me, so pick your poison.

First, I unregister all the locations, make the required modifications, and then reregister them. Then I redeploy all the dependent objects (mappings, process flows), and this worked fine. However, I only had two mappings and a single process flow. If this were an actual OWB project for a data warehouse, there would be lots of mappings and process flows, so you would have to decide if you think this is worth it.



The second choice is to use the Repository Browser to upgrade the locations, and the instructions are in the documentation. I start the Repository Browser listener by executing $ORACLE_HOME/owb/bin/unix/startOwbbInst.sh, and then point a browser to https://localhost:8999/owbb/RABLogin.uix?mode=design. After logging into the Repository Browser, I choose Location Reports. I can start with any of the database or process flow locations by clicking the “Unknown” hyperlink in the the Validation column.



I set the port number to 1522, and then click Update Details. Then I click Get Status, and click Upgrade.



This completes the upgrade of the locations. Now it’s time to execute the process flow, which in turn executes the two mappings.



So at this point, I have my OWB project migrated and working, but I’m not taking advantage of any of the new features that Mark has been blogging about. I’ll tackle that with my next post… how to modify mappings to make better use of the Code Template functionality.

]]> http://www.rittmanmead.com/2009/10/29/upgrading-an-owb-project-to-11gr2/feed/ 0 http://www.rittmanmead.com/2009/10/18/publishing-mappings-as-web-services-using-owb11gr2/ http://www.rittmanmead.com/2009/10/18/publishing-mappings-as-web-services-using-owb11gr2/#comments Sun, 18 Oct 2009 18:38:17 +0000 Mark Rittman http://www.rittmanmead.com/2009/10/18/publishing-mappings-as-web-services-using-owb11gr2/ Over the past few weeks I’ve covered some of the new features in OWB11gR2 including heterogeneous sources and targets, code template mappings, hybrid mappings, change data capture and OBIEE integration. 11gR2 comes with a bunch of other interesting new features including data chunking (automatic parallelization of PL/SQL mappings), orphan management for dimension loading, support for EJBs in process flows and enhanced configuration management, but one feature I particularly wanted to blog about was the ability of OWB11gR2 to publish and consume web services.

This feature works in two ways. Mappings, process flows, data auditors and transformations can be published as web services, as well as tables (more on this later on). Process flows can also now consume web services, so that data can be brought in from an external source and copied in to staging tables. The process works through publishing the mapping, process, data auditor or transformation to either OC4J Standalone or Oracle Application Server (though curiously, not Weblogic which is now the standard for the rest of the Oracle BI stack), so that it can either be called from an external process or a remote installation of Warehouse Builder. So how does it work?

To keep things simple, I created a basic mapping that copied a few rows of data from a source table to a target table. The web service integration works with both classic and code template mappings, and this one was created as a straightforward classic (database-resident) mapping.



I then deploy this mapping to the target database as usual.



Now it’s time to set the web service up. To start, I create a new Application Server module within the OWB11gR2 Project Explorer, a module type that’s new for this release.



I connect to the bundled OC4J instance that comes with OWB11gR2, and enter the connection details.



Once this module is set up, I can see it in the Project Explorer along with entries for individual web services and web service packages.



At this point I can add two types of web service. I can register external web services, via a WSDL file and a URL, so that I can reference them in process flows within my project. Or I can do what I’m going to do now, and register one of the mappings, process flows, transformations or data auditors as a web service, so that I can call them from outside Warehouse Builder or indeed through one of my own process flows (I’ll explain why in a moment).

The simplest way to make a mapping available as a web service is to right-click on it and select Create as Web Service. I do this with the mapping I created earlier.



This brings up an initial dialog where I select the application server to deploy it to. I select the local OC4J instance that I registered in the project earlier.



Warehouse Builder creates the new web service in the application server module. The project now looks like this.



Double-clicking on the new web service shows the options for web services within OWB. For this entry though, the choices are greyed out and all I can do is view what’s already been set.



Creating a new web service from scratch though, without right-clicking on an existing module, makes all of the options available. Interestingly, there’s an option to Create from a Table or Module for Change Data Capture.



In Oracle Data Integrator, creating a web service over an existing table or view makes it available for insert, update, select or delete via web service calls (see here for more information on ODI and web services). In OWB11gR2 though, creating web services off of tables or database modules provides methods for starting and stopping changed data capture, extending and purging the capture window, or adding or removing subscribers, which answers the question in my previous blog post on change data capture as to how these tasks are performed programmatically. What presumably you’d do then is publish these web service methods and then call then from within a process flow to extend and purge the CDC subscriber window, a bit of a roundabout method but one that replicates the module functionality in ODI.

Going back to my previous web service, I now need to generate the code to implement the web service. I select the web service module and select Generate….



This generates the code and reports back on the success or failure of the task.



Everything looks OK so far. I then deploy the web service, again by right-clicking on the entry in the Project Explorer and selecting Deploy… (I could have used the Control Center Manager instead, if I liked).



Again the task seemed to execute properly, with no reported errors.



Before you can run a web service built using an OWB mapping, you need to create an OC4J user that you then assign an execute privilege to. You do this from the main Design Center menu, selecting J2EE User Management.



Using this interface I create a new user, and assign it the OWB_J2EE_EXECUTOR role, which comes pre-created in the bundled OC4J instance that comes with OWB11gR2, but which you’ll need to create beforehand in standalone installations.



So now I’m ready to roll. I could execute the web service from a URL, or from the OC4J Console that comes with the OC4J instance, like this:



But the easiest way to test it is to run it from within the Control Center Manager, where I can just right-click on it and select Start.



Starting the web service brings up a menu where i can select the method to be invoked. I select the Run method and press OK.



The web service then runs and completed successfully.



Finally, checking the contents of the target table, the rows from the source table have been loaded as expected.



Consuming a web service is a fairly straightforward process, and a web service called in this way would typically either move data into a staging area, transform some data or in the case below, extend the subscriber window for a CDC operation.



When you add a web service invocation to a process flow, you are presented with a list of the methods available. Once you’ve selected the method you are interested in, the web service then appears in the process as just another task giving you an easy way to call external processes or perform administration around change data capture.



So there you have it. If you happen to be in Helsinki on 5th November, or at our Training Days later in October, I’ll be going through all of these new features in a lot more detail. Until then, if you get to use anything new in OWB11gR2 on a project, be sure to leave some feedback, and when things are a bit quieter around here I’ll finish off the series by looking at data chunking and orphan management.

]]> http://www.rittmanmead.com/2009/10/18/publishing-mappings-as-web-services-using-owb11gr2/feed/ 0