Simple Version Control for OBIEE using Subversion, VisualSVN Server and TortoiseSVN

January 22nd, 2009 by

A question I’ve been asked fairly regularly over the past few weeks is how you can version control the metadata used by an OBIEE project. As you’re probably aware, there is no real version control built in to OBIEE (you can check-in, and check-out, repository objects in the BI Server repository when it’s running online, but this is more to make sure that two people don’t edit the same repository object at the same time). What we really need is a way to take the various files that make up an OBIEE installation and save them into a version control system, saving copies of each revision and allowing us to access a specific version, or “release” of our OBIEE system so that we can refer back to it, or deploy it into another environment such as production.

Now you can make version control as complicated as you like, but to illustrate the principals what I thought would be useful would be to set up a simple version control system on an all-Windows environment using VisualSVN Server, a free “packaging up” of the open-source Subversion version control system, and the free TortoiseSVN Windows Explorer extension that provides Windows integration with Subversion. Once you’ve got them both installed (VisualSVN usually on a server, TortoiseSVN on your developer PC) you can make a start.

The first step is to start the VisualSVN Server management tool and create a new repository. I’ve called my repository “OBIEE”, and I selected the option to create the default Subversion directory structure. “Trunc” is where your project files will go; “Branch” is for when you create branches of your code (independent code streams that had a common root); “Tags” is where you can copy complete copies of your “Trunc” directory when you want to specifically save a release of your code.

Svn Obiee1

Once you’ve created the base repository and the top-level trunc, branches and tags directories, you can now create directories and subdirectories under the trunc directory for the project. In my case, I create directories for the main parts of an OBIEE and Oracle BI Apps project, including the reports, metadata and product configuration files.

Svn Obiee1A

So now that the project folders are created in Subversion, I can check these out and copy them, as my “working copy”, to the filesystem. The idea then is that I use this as the gathering point for all my project files, uploading them again to Subversion when I’m ready to check in a release of the code.

To check out a working copy of the code, I right-click anywhere on the Windows desktop and select SVN Checkout… from the context menu.

Svn Obiee3-1

Selecting this option then brings up a Checkout dialog, where I can choose to check out the latest version of the project (the “HEAD” revision), or a specific revision of the project. For now I just select the HEAD revision as there’s nothing else other than the folders in the repository anyway.

Svn Obiee4

Once I select my revision, TortoiseSVN then shows me the various project files downloading to my desktop from the Subversion server.

Svn Obiee5

So if I now want to package up a specific release of my OBIEE and Oracle BI Apps project, it’s a case of copying the various project files to the new set of folders that have been copied to my desktop. Taking a look at the folders, there are directories for the RPD repository file, the Web Catalog, the various BI Server, BI Presentation Server, BI Publisher and JavaHost services, plus directories for DAC exports and backups of the Oracle_BI_DW_Base Informatica repository.

Svn Obiee5A

Now that there’s a local filesystem to copy the various files to, the next job then is to go and get all the relevant files and place them in the correct directories prior to upload. This table sets our where the files are (this list isn’t exhaustive, if you know of any others then let me know) and where they need to get copied to in the working directory.

Svn Obiee15

Once the files are all in place, you can then use TortoiseSVN to upload, or “commit” them to the Subversion repository. To do this, right-click on the top-level local folder and select SVN Commit... from the context menu.

Svn Obiee6

You are then presented with a dialog that lets you determine what is committed. In my case I select all the files so that the entire state of the project is uploaded to the repository.

Svn Obiee7-1

After pressing OK, the project files are then uploaded to Subversion. This can take a few minutes if you’re uploading the entire Web Catalog, for example, though subsequent uploads should be quicker as you’ll only be uploading the new and changed files.

Svn Obiee8

So now that you’ve got the project files uploaded to the repository, you can go ahead and upload further revisions later on, with all of the files being versioned and saved within the repository, or you can use the SVN Checkout… feature to download either the latest version, or a specific version, of the project for use in another environment.

Svn Obiee9

Each upload of project files to the repository generates a new revision number; these can be viewed from the Checkout dialog when you make your checkout selection.

Svn Obiee11

So that’s the basics for an OBIEE project. You set the folders up initially in Subversion, then copy the files down to the local filesystem to create your “working copy”. You then place all the various OBIEE files into this local filesystem, and upload its contents to Subversion every time you want to record a version. As a process it could be improved; it would be nice if you could upload the project to Subversion from within Oracle BI Administrator, and it’d also be nice if you could avoid having to copy all the files to an upload location and instead leave them in place in the OracleBI and OracleData directories, calling an “upload” function from somewhere that uploaded them in place. It’s also a fairly coarse-grained way of versioning the RPD repository file, and you might want to consider exporting the repository to a UDML file instead and uploading that, as a text file, to Subversion so that you can perform diffs on it and perhaps branch the code. But it’s a simple, basic way of adding some sort of version control to your OBIEE system and at least being able to refer back to past versions of what you’ve done.

If you’re working with the Oracle BI Applications, you can also copy the contents of the DAC repository (via the DAC Export feature) into this filesystem, as well as an export of the Informatica repository. To export the DAC contents, log on to the DAC Console and select Tools > DAC Repository Management, select the folders you want to export and then copy the exported XML files to the /DAC folder in the Subversion working directory.

Svn Obiee12

If you want to export your Informatica repository and you’re on version 7.9.5, fire up the Informatica PowerCenter Administration Console and locate the entry for your repository server, then select Actions > Back Up Contents from the menu to bring up the Repository Backup menu.

Svn Obiee13

You can then pass in the username and password for the repository, as well as the path to the Subversion working directory and the backup filename, and this should create a copy of the repository that you can subsequently import back into an Informatica environment.

Svn Obiee14

So there you have it. Of course there’s a lot more to Subversion than just this; you can create branches of your code, and you can create copies of your project under the “tags” directory to save specific sets of files as a particular release. But it’s a good start and not too onerous to set up, and something we use on projects as a low-impact way of adding source control to an OBIEE project.

Comments

  1. mihai.onofreiciuc Says:

    Hi Mark,

    I do the version control in different ways for the plain text and for the binaries files.

    The use of SVN is appropriate for text files.
    I am scheduling the SVN commit to run 3-4 times a day. Beneath OBIEE configuration text files, it is worth adding also the database exported DDL here.

    For binaries, I create a daily scheduler which will create big archives. I keep last 7 days archives + 1 archive per month this year + 1 archive per quarter previous years.

  2. Mark Rittman Says:

    Hi Mihai

    Some good feedback, thanks. With regards to the repository (RPD) files, plus other binary files such as the Informatica Repository backup, the reason I bring them in to Subversion along with all the other files is so that I can tie them all together as a consistent “release”.

    What you’re suggesting (regular inclusion in an archive file) is great for backups but it doesn’t solve the issue of creating point in time consistent sets of OBIEE metadata. As far as I’m aware, Subversion should detect that the RPD is a binary file and not try to merge changes in to it, so there shouldn’t be any danger in this approach (though I’m willing to be corrected on this if I’m wrong).

    Is there any specific reason why you /wouldn’t/ upload the OBIEE binary files into Subversion? Maybe the size of them, as it’s one big file and you can’t upload just the changes (as you can with the DAC export, or the Web Catalog?)

    regards

    Mark

    reg

  3. John Minkjan Says:

    Mark,

    Just to add to your list:
    the S_ and SK_ directorys you are customizing.

    regards John

  4. Sid Says:

    Mark,
    Actually there are advantages to Mithia’s suggestions. If you store the textual version of the repository (the UDML version through the conversion utility or copy) into the repository then you can compare two versions of the repository in text mode and you can see the names of the rpd objects added even if one does not know the UDML (who knows anyway?). SVN (or any other source code control tool like PVCS) are not very sophisticated in merging changes in complex scenarios and therefore it is not correct to depend on the merge process.
    Adding the files to SVN is also beneficial in the sense that deployment can be automated through the scripts.

  5. Sid Says:

    Mark,
    Forgot to say that this is a beautiful article. Keep up the good work, learned a lot from you.

  6. Pete Finnigan Says:

    Hi Mark,

    Thanks for the great article, I have a few people I know I can point to it straight away. I get asked similar questions regarding sourcfe control but for security configurations in the database and for key database config files. I have suggested Visual SVN Server for some time now and also Tortoise. I also looked at the add-in for Visual Studio for our own internal development (from the same people as the free server). Nice article.

    cheers

    Pete

  7. Mariano Says:

    Thanks for this article, it’s great for starters.

  8. kamlesh Says:

    Hi Mark,

    is there any new tool from Oracle which integrates in OBIEE to track/migrate for Version control as well to track the migration of objects ?

    thanks,
    Kamlesh

  9. Mark Rittman Says:

    Kamlesh

    Not that I am aware of. Oracle must internally have some sort of system for when they are developing the RPD for the BI Apps, but nothing has been released externally. Most clients I know use some variation on the Subversion approach outlined in this article.

    regards, Mark

  10. swetha Says:

    Hi Mark,

    We have our webcatalog on one linux server and subversion on another server. I have master copy of webcat in subversion and loaded it into my local system. Is it possible to make changes in reports (offline) in local system and upload it to subversion.Is there way to get online webcat changes into subversion directory

    Your help is appreciated.

    Thanks,
    Swetha

  11. Ashish Says:

    See my latest blog for a way to get compare and merge rpd files from tortoiseSVN.

    navi-obiee.blogspot.com

  12. Mamta Says:

    Hi Mark,

    RPD is not recongnised as binary file in SVN.IF there are any conflicts,how to resolve them?

  13. Ross Morrissey Says:

    Thanks for the helpful article. Are there any significant changes for 11g?

  14. Guy De Paepe Says:

    Hi Mark,

    Indeed great article. SVN is a great tool for version control, but the issue we are facing is that we have several changes pending between our DEV, UAT and PROD environments. Which means, if you migrate your RPD from DEV to QA for instance, you migrate all pending changes, including the ones that have not yet been tested and approved. Any suggestions on how to handle this scenario ?

    Greatly appreciate your comments.

    Thanks

    Guy

  15. amol palkar Says:

    nice one Mark !!!

Website Design & Build: tymedia.co.uk