Using Groovy instead of WLST for OBIEE Systems Management

February 17th, 2014 by

I remember some years back when Rittman Mead first received the OBIEE 11g beta and I tried installing it for the first time. It was a nightmare. If you think the 11.1.1.3 release was challenging, you should have tried working with any one of the betas. It was with these first few releases that Weblogic was injected into my world. Some time around the first month of starting the beta, I recall Mark Rittman saying something along the lines of: “all the Fusion Middleware stuff looks interesting, but I’m glad we won’t have to know anything about that.” You see… I just wanted a nice career in BI, but here I am writing a blog post on working with JMX MBeans.

JMX MBeans are complicated, meaning that careers probably exist for working with JMX MBeans and little else. I’m truly sorry to hear that. For the layperson (and by this, I mean people who work in BI), trying to understand MBeans probably ranks somewhere between getting decent service at a restaurant in the UK, and understanding why the Anomaly needs to join with the Source. It’s complex enough that most vendors provide utilities to simplify the process… which means using anything other than Java to work with them. For Weblogic, we have Weblogic Scripting Tool (WLST).

WLST is really just a specific implementation of Jython designed into a command-line utility that in many ways feels like SQL-Plus or RMAN. It’s designed to work interactively or in scripted fashion, and Rittman Mead has offered up some free WLST scripts to the community over the years via GitHub. If you take a look at our deploy_rpd.py script for example, you can make some sense of how WLST works. I’ve reworked that script slightly in this post to use hardcoded values instead of variables, to remove the exception handling, etc., to make the logic easier to follow. It’s shown here in two parts… the first part which connects to the Weblogic Administration Server and locks the domain:

This lock method is equivalent to the “Lock and Edit” button inside Fusion Middleware Control. We use the cd command to navigate to the BIDomain MBean because the lock method (as well as commit and rollback ) is in that MBean. The invoke command is used to execute MBean methods in WLST and accept parameters for those methods. To do this we use generic Object and String arrays because the invoke command is generic and needs to support an ever-changing series of parameters. The rest of the pared script is below:

In the second part of this script, we navigate to the ServerConfiguration MBean, which contains the uploadRepository method. We populate the params array with the RPD path and RPD password, we upload the repository, and then navigate back to the BIDomain MBean to perform the commit.

WLST strikes me as an attempt to take a programmatic interface, one that is usually accessed through Java, and make it accessible to non-developers. In this regard, I see it as a major failure. The MBean methods have been abstracted to the point of being almost useless. This seems like the exact opposite of simple to me. It seems overly complex to have to construct arrays and populate them just to pass parameters to methods. I would prefer to just pass parameters as, well… parameters. Additionally… notice how we had to cd to the BIDomain MBean, cd away to go use the ServerConfiguration MBean, and then cd back again. I would prefer to use the MBeans the way they were meant to be used… as objects, so we would be able to instantiate and hold multiple MBean objects at the same time.

If WLST is indeed easier to use than Java, then this is quite an indictment of Java. It works pretty well when we are writing processes that are JMX specific… when the commands that we need to issue are all calls to MBeans and nothing else. But what happens if we want to write a process that involves interacting with MBeans as well as some other processes or APIs, such as calls to any of the Oracle BI Server XML API, in a unified script? WLST is terrible at this. In my first attempt at writing something like this, I used Perl to execute both validaterpd commands as well as MBean commands. I ended up having to maintain a separate Jython script along with the Perl script, which didn’t sit well with me as a developer. What we really need is a language that is groovy to write and maintain, compiles to pure Java (just like Jython), and has the ability to encapsulate MBeans natively in a similar fashion to WLST. What we need is Groovy.

Groovy is a dynamic language that can be used to write scripts, or write object-oriented packages and classes, and can also be used to compile to Java byte code. Because it’s simple to write and has native MBean object handling, it’s easier (in my opinion) than WLST. How easy? Well, let’s go through our deploy_rpd.py example, and see if it’s easier to write and understand. Here’s our upload repository script in Groovy:

That’s the whole script. There’s a few things to point out here. First… establishing connectivity to the Weblogic Admin Server is slightly harder in Groovy. WLST hides a good bit of the complexity with connectivity, but the JMXConnectorFactory class in Groovy is a close second. However, once the connectivity is done, Groovy wins the rest of the way. Notice how we are able to configure two different MBean objects: biDomain for BIDomain and servConfig for ServerConfiguration. We don’t have to cd back and forth across the MBean hierarchy to work with the different MBeans; we can simply instantiate as many MBeans at once as we need to.  Also, notice that we don’t have the generic invoke command to call the different methods in MBeans. We just call the method directly: if we want to call lock(), or commit(), or uploadRepository(), we just call it, and pass the parameters that it accepts. No need to build the generic Object and String arrays for passing in to invoke.

Because Groovy is very easy to execute external command-line processes (such as biserverxmlexec or biserverxmlgen) it’s a great choice for unified OBIEE processes that need to work with executables and JMX MBeans together in a single script. Also… Groovy accepts Java syntax, so if we can’t find the Groovy code samples we need from the internet, we can just plug in Java samples instead.

Comments

  1. Mark Rittman Says:

    At least our plugs don’t fall out of the socket by themselves, when you’re trying to iron a shirt in an American hotel room.

    And as for service – http://www.youtube.com/watch?v=7avWjBmRXu0

  2. Girish Lakshmanan Says:

    Nice article Stewart.

    I’m salivating at the thought of MDS XML + Groovy + XMLSlurper + GPath.
    The possibilities are endless.

    However, I probably wouldn’t use Groovy for pure magagement services. I think that WSLT provides better overall abstraction than Groovy as a JMX client. There may be a couple of advantages of using Groovy but I think WSLT is more abstract and therefore easier to use.

  3. Stewart Bryson Says:

    @Girish:

    Thanks for reading, and for commenting. I would say that WLST has it’s merits, but the real kicker for Groovy is the ability to write class files in Groovy that contain methods for making JMX connectivity, constructing MBeans, uploading repositories, etc. I wrote a Groovy class files that handles establishing connectivity, that automatically instantiates and stores all the key MBeans, etc. So now, my code for uploading a repository looks like this:

  4. Nicolas Gerard Says:

    @All
    The question of what programming language you will use is just related to a couple of choice.
    – the knowledge that have your team
    – in which kind of language your prefer to work ie typed (Java, C) vs not typed (Python, Groovy, …)
    – the performance
    – the community (ie the libraries available)
    – the ease of use (Ide,…)

    But the most important is to do it.
    @Stewart Well done.

    Cheers
    Nico

  5. Girish Lakshmanan Says:

    Ofcourse Nico.

    The idea that Stewart floats is great – which is to build a custom JMX client aimed at OBIEE administration – abstracting a few mundane tasks along the way.
    Comparing Groovy to WSLT in terms of ease-of-use however is slightly apples and oranges ?
    We are comparing an existing JMX client to building a new one using a programming language(which in this case is Groovy).
    There is no question that an existing JMX client will be easier to use than build a new one ? WLST as a JMX client is specifically aimed at Weblogic’s way of organising its Mbeans(in a filesystem like tree). Would be quite a task to write a custom client to match it with the breadth of functionality.
    WLST also has other avatars such as allowing offline access to weblogic config – which is outside the realms of a JMX client. Anyways, that is not of much interest to OBI specific systems management.

    Having said that, I’m all for the idea of abstraction aimed at OBIEE environments(developing a custom JMX client with Groovy or Java). It would certainly benefit all of us. In my head I can see a nice OBI specific product with Systems management features, Change control features etc.

  6. Stewart Bryson Says:

    @Nicolas: Thanks for taking the time to comment. I agree with your points. Just one clarification: Groovy can be typed or not-typed. I tend to use types as a way of self-documenting, but in these examples I used the “def”, which is not-typed.

    Thanks again Nicolas.

  7. Nicolas GERARD` Says:

    @Girish I understand better your point of view and you are totally right.

    @Steward Does your Ide (Eclipse, …) give you – a list of function when you type a point after an object. Such as after jmxConnector ?
    – the compile error before you run your groovy code ?
    If not, it’s what I call a non-typed language. A better word was dynamic type of may be high level language with an interpreted compiler …

  8. Chetan Kedia Says:

    Hi Stewart,

    This is nice article. Best will be to code in java. With java the advantage is that we get the method name by just typing dot after the object name.With groovy we need to know the method name.

    Regards,
    Chetan

  9. Stewart Bryson Says:

    @Nicolas, Chetan:

    Using Eclipse (with the Groovy plugin) does in fact provide autocompletion (a popup after typing dot) and shows all the Java equivalent attributes and methods in the Project and Package browser. I can write a Groovy method using default parameter values, and Eclipse automatically shows me the multiple Java methods that this would compile to. It also gives compile errors as you are typing, just like Java. See the features here: http://groovy.codehaus.org/Eclipse+Plugin#EclipsePlugin-FeaturesandStatus

    Groovy is just a “preprocessor” language… what you get on the other end is Java. It’s a shortcut… using a simpler language to “auto-generate” Java code for you. See this article: http://www.javaworld.com/article/2084582/scripting-jvm-languages/15-hot-programming-trends-and-15-going-cold.html#twitter

  10. Naeem Akhtar Says:

    Very interesting article. Little learning of groovy can save lot of time for the administration .

  11. Sid Says:

    Yes, WLST has it’s limitation but there is one crucial problem in adopting groovy to manage fusion middleware. Lack of documentation on which JMX method to use is a big problem. When I have thousands of WLST script found around Internet and I would not even know the place to go to find the methods to use with groovy, I would not find typing few extra lines very bothersome.

    I typically use jython to do top level processing, build a temp script and invoke WLST to do the we logic stuff.

  12. Stewart Bryson Says:

    @Sid: Pragmatism is always a powerful reason to choose one path or another. I believe that was the main point that Nico was making when he wrote that a team’s strengths are just as important when choosing a language as the value of the language itself. Being able to find an repurpose WLST scripts on the internet is a valid reason to stay with WLST, and when the extent of what you find is sufficient, then by all means, keep scripting.

    My main point with this post was that WLST does not work well when we are developing a process that is only one part JMX Bean interrogation. In these cases, I would argue we need a more powerful language. Also… the version of Python that WLST uses is ancient… so trying to utilize third-party Jython/Python classes is a fools errand in WLST.

    As far as documentation of JMX Beans… we really can’t get much better than the MBean Browser. It allows us to graphically test the attributes and methods in a browser before writing a line of code. I have to disagree on this point… I don’t think that one holds water. Sorry.

    http://wp.me/a4yr8-4Iw

    http://wp.me/a4yr8-4Ix

    @Nico: hopefully I didn’t put words in your mouth.

  13. Nicolas Gerard Says:

    Unfortunately, I was this last week a little bit off-line.

    I wanted to test Groovy before giving a reaction. And indeed the datatype is recognized even if the variable name stay fix. (With Intellij). Impressive.

    It likes to be as you say a language builds on top of Java. While debugging, the variables are saved in an Linkedlist structure. It’s not really convenient to read them.

    I agree with you with the fact that you must be pragmatic.

    “Make it first, then make it right, then make it faster” is my motto.

    Scripting language such as Groovy doesn’t control the data type and it’s something that with the experience I didn’t like when my program gets bigger and bigger.

    In all cases, I need to improve my scripting skills in order to stay up to date and to see new techniques. I will be waiting an ODI project ;-)

    Thanks and good weekend
    Nico

Website Design & Build: tymedia.co.uk