Comments on: Using a Process Flow to loop through Months in a Table http://www.rittmanmead.com/2009/07/28/using-a-process-flow-to-loop-through-months-in-a-table/ Delivered Intelligence Fri, 30 Jul 2010 17:54:00 -0400 http://wordpress.org/?v=2.8.4 hourly 1 By: Stewart Bryson http://www.rittmanmead.com/2009/07/28/using-a-process-flow-to-loop-through-months-in-a-table/comment-page-1/#comment-10425 Stewart Bryson Thu, 30 Jul 2009 02:48:13 +0000 http://www.rittmanmead.com/?p=2594#comment-10425 @Tim: Thanks for reading, and for commenting. When in doubt, PL/SQL works. It's not usually considered a BI skill, except by the people in BI who use it. @Tim: Thanks for reading, and for commenting. When in doubt, PL/SQL works. It’s not usually considered a BI skill, except by the people in BI who use it.

]]> By: Tim Berry http://www.rittmanmead.com/2009/07/28/using-a-process-flow-to-loop-through-months-in-a-table/comment-page-1/#comment-10422 Tim Berry Wed, 29 Jul 2009 15:53:16 +0000 http://www.rittmanmead.com/?p=2594#comment-10422 Thanks for the update I would say that the saving grace for OWB is the ability to integrate bespoke pl/sql. There are always circumstances where simple set based logic does not suffice and OWB resolves this requirement well with the simple integration. Thanks for the update I would say that the saving grace for OWB is the ability to integrate bespoke pl/sql. There are always circumstances where simple set based logic does not suffice and OWB resolves this requirement well with the simple integration.

]]> By: Stewart Bryson http://www.rittmanmead.com/2009/07/28/using-a-process-flow-to-loop-through-months-in-a-table/comment-page-1/#comment-10410 Stewart Bryson Tue, 28 Jul 2009 12:03:57 +0000 http://www.rittmanmead.com/?p=2594#comment-10410 @Tim: I try to do as much within OWB as possible, and usually find I don't require control tables to drive flow logic. Rittman Mead does often use a control table to manage mappings, but this is merely for rerunability... doing a quick check to see if the mapping has already run in a restart scenario. @Tim: I try to do as much within OWB as possible, and usually find I don’t require control tables to drive flow logic. Rittman Mead does often use a control table to manage mappings, but this is merely for rerunability… doing a quick check to see if the mapping has already run in a restart scenario.

]]> By: Tim Berry http://www.rittmanmead.com/2009/07/28/using-a-process-flow-to-loop-through-months-in-a-table/comment-page-1/#comment-10409 Tim Berry Tue, 28 Jul 2009 10:44:49 +0000 http://www.rittmanmead.com/?p=2594#comment-10409 A very common requirement for ETL processing and a nice solution. Usually there is a table driving the process which is populated with a variety of means to determining the set required (key values, date ranges etc). Unfortunately there maybe a need for inspecting the volume of the driving set to influence the processing and subsequently if X is set i.e. low volume then use insert Y and if X is not set i.e. high volume then use insert Z. Sometimes you just cant get away from it but at least the two execution paths are alongside one another. A very common requirement for ETL processing and a nice solution. Usually there is a table driving the process which is populated with a variety of means to determining the set required (key values, date ranges etc).

Unfortunately there maybe a need for inspecting
the volume of the driving set to influence the processing and subsequently if X is set i.e. low volume then use insert Y and if X is not set i.e. high volume then use insert Z.

Sometimes you just cant get away from it but at least the two execution paths are alongside one another.

]]> By: Peter Scott http://www.rittmanmead.com/2009/07/28/using-a-process-flow-to-loop-through-months-in-a-table/comment-page-1/#comment-10408 Peter Scott Tue, 28 Jul 2009 07:59:37 +0000 http://www.rittmanmead.com/?p=2594#comment-10408 Stewart - I have used something similar with the aggregation build for a customer's DW - here we used hand-refreshed MVs for the summary tables which were partitioned by time - as part of the daily load we inserted into a control table a list of distinct dates to process at the successful conclusion of the batch the control table was truncated - if we needed to re-summarise a day we added it to the control table, to rebuild all of the data warehouses we just added all of the dates to process Stewart – I have used something similar with the aggregation build for a customer’s DW – here we used hand-refreshed MVs for the summary tables which were partitioned by time – as part of the daily load we inserted into a control table a list of distinct dates to process at the successful conclusion of the batch the control table was truncated – if we needed to re-summarise a day we added it to the control table, to rebuild all of the data warehouses we just added all of the dates to process

]]>