Comments on: Oracle BI EE 10.1.3.4.1 – Sub-Totals & Pivot Calculations http://www.rittmanmead.com/2009/11/17/oracle-bi-ee-10-1-3-4-1-sub-totals-pivot-calculations/ Delivered Intelligence Wed, 17 Mar 2010 13:47:41 -0400 http://wordpress.org/?v=2.8.4 hourly 1 By: Venkatakrishnan J http://www.rittmanmead.com/2009/11/17/oracle-bi-ee-10-1-3-4-1-sub-totals-pivot-calculations/comment-page-1/#comment-11281 Venkatakrishnan J Thu, 26 Nov 2009 10:26:15 +0000 http://www.rittmanmead.com/?p=3655#comment-11281 @Harry - Well i have seen the same case even with BI EE implementations as well. The problem is end users are comfortable with pivot tables since they are similar to Excel. The example that you have pointed out where we require sub-totaled measures by Time (wks or months) etc are all possible using custom Sum/Max/Min/etc By calculations even in a normal table view. The approach i always use is to start with table view. I think of all possible options to create the reports using Table View itself. If the report layout is so complex that a table view cannot be used, only then will i go in for a Pivot View. @Harry – Well i have seen the same case even with BI EE implementations as well. The problem is end users are comfortable with pivot tables since they are similar to Excel. The example that you have pointed out where we require sub-totaled measures by Time (wks or months) etc are all possible using custom Sum/Max/Min/etc By calculations even in a normal table view. The approach i always use is to start with table view. I think of all possible options to create the reports using Table View itself. If the report layout is so complex that a table view cannot be used, only then will i go in for a Pivot View.

]]> By: Harry M http://www.rittmanmead.com/2009/11/17/oracle-bi-ee-10-1-3-4-1-sub-totals-pivot-calculations/comment-page-1/#comment-11280 Harry M Thu, 26 Nov 2009 08:09:52 +0000 http://www.rittmanmead.com/?p=3655#comment-11280 "As a generic rule, in any implementation, if the repository is correctly designed, almost 90 to 95% of the reports can be achieved using normal table views" I have seen the opposite, whereby 95% of reports were pivottables (this was not using OBIEE). The customer required a view of all offices, subtotalled by region BY time i.e weeks or months. So a Pivottable was required for each measure(report) with Region and Office down the left and Time across the top. We are considering using OBI but it seems we will hit performance issues unless this type pf report can be created as a table or we use OLAP? “As a generic rule, in any implementation, if the repository is correctly designed, almost 90 to 95% of the reports can be achieved using normal table views”

I have seen the opposite, whereby 95% of reports were pivottables (this was not using OBIEE). The customer required a view of all offices, subtotalled by region BY time i.e weeks or months. So a Pivottable was required for each measure(report) with Region and Office down the left and Time across the top. We are considering using OBI but it seems we will hit performance issues unless this type pf report can be created as a table or we use OLAP?

]]> By: Venkatakrishnan J http://www.rittmanmead.com/2009/11/17/oracle-bi-ee-10-1-3-4-1-sub-totals-pivot-calculations/comment-page-1/#comment-11273 Venkatakrishnan J Wed, 25 Nov 2009 09:35:29 +0000 http://www.rittmanmead.com/?p=3655#comment-11273 @Steve - It is very much possible that Pivot tables can generate another physical SQL for calculating the Sub-Totals. This is to an extent controlled by the "Report Based Sub-totals wherever possible" setting. Unfortunately we cannot exactly say when BI Server will switch between temp file based sub-totals to another physical SQL when the above setting is turned on. But in most cases, we can control whats happening through a combination of repository design and report level aggregations. @Steve – It is very much possible that Pivot tables can generate another physical SQL for calculating the Sub-Totals. This is to an extent controlled by the “Report Based Sub-totals wherever possible” setting. Unfortunately we cannot exactly say when BI Server will switch between temp file based sub-totals to another physical SQL when the above setting is turned on. But in most cases, we can control whats happening through a combination of repository design and report level aggregations.

]]> By: Steve Devine http://www.rittmanmead.com/2009/11/17/oracle-bi-ee-10-1-3-4-1-sub-totals-pivot-calculations/comment-page-1/#comment-11272 Steve Devine Wed, 25 Nov 2009 09:21:57 +0000 http://www.rittmanmead.com/?p=3655#comment-11272 I've also noticed that OBIEE generates a 2nd physical SQL statement just to calculate the Pivot table totals. This causes problems in cases such as showing the true Sales Value / 1000000 for instance. Due to rounding the pivot table rows will never reconcile with the Total. You can obviously debate that they are both correct but from a user perspective it is totally confusing. Any ideas why OBIEE sometimes generates a second query v's the approach you describe above? Oracle Support can't help! I had a pivot table which did not generate 2 queries, I copied it to a new Request and made one or 2 small mods and the new one started generating 2 queries! I’ve also noticed that OBIEE generates a 2nd physical SQL statement just to calculate the Pivot table totals. This causes problems in cases such as showing the true Sales Value / 1000000 for instance. Due to rounding the pivot table rows will never reconcile with the Total. You can obviously debate that they are both correct but from a user perspective it is totally confusing. Any ideas why OBIEE sometimes generates a second query v’s the approach you describe above? Oracle Support can’t help! I had a pivot table which did not generate 2 queries, I copied it to a new Request and made one or 2 small mods and the new one started generating 2 queries!

]]> By: Venkatakrishnan J http://www.rittmanmead.com/2009/11/17/oracle-bi-ee-10-1-3-4-1-sub-totals-pivot-calculations/comment-page-1/#comment-11242 Venkatakrishnan J Wed, 18 Nov 2009 08:49:14 +0000 http://www.rittmanmead.com/?p=3655#comment-11242 @Kimmo - Thats correct. I have seen temp files going in the range of 3 to 5 GB with a single user working on a set of pivot tables. Unfortunately there is no compression that BI EE does while writing to the disk. Apart from compression, this I/O activity can take significant amount of time while working on a set of big pivot tables. This is in addition to the SQL time. Thats the reason why i always recommend restricting the use of Pivot tables. The functionality in most cases can be achieved by using normal table view. -Venkat @Kimmo – Thats correct. I have seen temp files going in the range of 3 to 5 GB with a single user working on a set of pivot tables. Unfortunately there is no compression that BI EE does while writing to the disk. Apart from compression, this I/O activity can take significant amount of time while working on a set of big pivot tables. This is in addition to the SQL time. Thats the reason why i always recommend restricting the use of Pivot tables. The functionality in most cases can be achieved by using normal table view.

-Venkat

]]> By: Kimmo http://www.rittmanmead.com/2009/11/17/oracle-bi-ee-10-1-3-4-1-sub-totals-pivot-calculations/comment-page-1/#comment-11241 Kimmo Wed, 18 Nov 2009 08:45:38 +0000 http://www.rittmanmead.com/?p=3655#comment-11241 Forgot to say that OBIEE doesn't clear out the temporary files if the maximum file size was reached or there was some other error (e.g. user cancelled the query) when generating the temporary files. Files are only cleared when restarting the services or with your own custom scripts. BR, Kimmo Forgot to say that OBIEE doesn’t clear out the temporary files if the maximum file size was reached or there was some other error (e.g. user cancelled the query) when generating the temporary files. Files are only cleared when restarting the services or with your own custom scripts.

BR,
Kimmo

]]> By: Kimmo http://www.rittmanmead.com/2009/11/17/oracle-bi-ee-10-1-3-4-1-sub-totals-pivot-calculations/comment-page-1/#comment-11240 Kimmo Wed, 18 Nov 2009 08:42:15 +0000 http://www.rittmanmead.com/?p=3655#comment-11240 Hi Venkat, The temporary files can be really big (gigabytes) when working with complex large pivot tables. Another thing to note that it seems that when running Answers these temporary files are generated on the server where Presentation Services are located and when running Scheduler files are generated on the server where Scheduler is running. One small thing to consider when planning the architecture and disk sizes of each server. Then there is also a bug in Linux/Unix version of 10.1.3.4, which limits the temporary files to 2GB. More info can be found here for example: http://obiee-blog.info/bugs-and-issues/nqserror-46073-analytics-temp-files-cannot-be-larger-than-2gb/ BR, Kimmo Hi Venkat,

The temporary files can be really big (gigabytes) when working with complex large pivot tables. Another thing to note that it seems that when running Answers these temporary files are generated on the server where Presentation Services are located and when running Scheduler files are generated on the server where Scheduler is running. One small thing to consider when planning the architecture and disk sizes of each server.

Then there is also a bug in Linux/Unix version of 10.1.3.4, which limits the temporary files to 2GB. More info can be found here for example: http://obiee-blog.info/bugs-and-issues/nqserror-46073-analytics-temp-files-cannot-be-larger-than-2gb/

BR,
Kimmo

]]>