[Mondrian] .Children function not being evaluated natively (no joinwith the fact table)

wenjin huang wenjin.huang at gmail.com
Wed May 27 18:34:01 EDT 2009


Hi Julian,

I have created a issue
(*MONDRIAN-561*<http://jira.pentaho.com/browse/MONDRIAN-561>) for
it in the Pentaho's Jira site. The attached file has the test cases for the
problem.

http://jira.pentaho.com/browse/MONDRIAN-561

Thanks,
Wenjin


On Fri, May 15, 2009 at 2:03 PM, Julian Hyde <jhyde at pentaho.com> wrote:

>  Wenjin,
>
> I'm nervous about accepting your change, because I don't understand the
> current code and this makes it yet more complicated. As Matt says, this is
> a general problem, not just related to Children and Descendants, so any fix
> that looks for specific function names is probably wrong. I want to change
> this code - as described in my blog post - and extra complications will make
> it more difficult to refactor later.
>
> So, I'd like you to create a unit test for this problem and contribute
> that. I saw you alrady have a query against FoodMart  that reproduces the
> problem. Add one or two methods to NonEmptyTest. Also log a bug, and make
> the test cases dependent on 'if (Bug.BugMondrianXXXFixed) ...'. Then test
> against 3.1 to see whether the bug still exists.
>
> Post a patch that fixes your test cases to this list, and I will consider
> integrating it. Even if I don't accept the patch, your test case will give
> us something to aim for when we refactor to solve this problem the 'right'
> way.
>
> Julian
>
>  ------------------------------
> *From:* mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
> *On Behalf Of *Matt Campbell
> *Sent:* Friday, May 15, 2009 7:40 AM
> *To:* Mondrian developer mailing list
> *Subject:* Re: [Mondrian] .Children function not being evaluated natively
> (no joinwith the fact table)
>
>   The bug you identified when using the Descendants() function is more
> general than just in named sets, and is probably more general than just
> using Descendants().
> SqlConstraintFactory will return a SqlContextConstraint, even if the
> current context is not nonempty.  Executing the query you posted will result
> in a SCC being used when evaluating the tuple list, which causes native non
> empty behavior.  I think a SqlContextConstraint should only be used in a non
> empty context.
>
> For other MDX functions (like .Children and .Members) there is an upstream
> check for nonempty, which causes a null context to be passed to
> SqlConstraintFactory.  If SqlConstraintFactory finds a null context it will
> return a DefaultMemberContextConstraint, resulting in the desired set.
>
> I think the simplest solution here would be to add a check for non empty
> context in the SqlConstraintFactory methods.
>
>
>  On Thu, May 14, 2009 at 8:04 PM, wenjin huang <wenjin.huang at gmail.com>wrote:
>
>> Hi,
>>
>> Thank you for your reply.
>>
>> We did some testing to sync the nonEmpty flag for root.slicerEvaluator
>> with its parent/caller RoalEvalutor. We tried the change below
>> in RolapResultEvaluatorRoot.evaluateNamedSet() method. With this change, The
>> .Children call will only return Non Empty children when it is specified
>> inside of NON EMPTY set. Since this change only applies to .Children and
>> Descendants function, it doesn't break any unit tests so far.
>>
>> RolapResult$RolapResultEvaluatorRoot.evaluateNamedSet(String, Exp) line:
>> 1194
>>
>>         //Pass in nonEmpty flag from the caller/parent RoalEvalutor
>>         protected Object evaluateNamedSet(String name, Exp exp, *boolean
>> nonEmpty*) {
>>             Object value = namedSetValues.get(name);
>>             if (value == null) {
>>                 final RolapEvaluator.RolapEvaluatorRoot root =
>>                     slicerEvaluator.root;
>>                 final Calc calc = root.getCompiled(exp, false,
>> ResultStyle.LIST);
>>
>>                * //Sync the nonEmpty flag for root.slicerEvaluator with
>> its parent/caller RoalEvalutor
>>                 //Restrict the change only to .Children call and
>> Descendants function.
>>                 //This flag will be used in the
>> DescendantsFunDef.descendantsByDepth() method for Descendants function,
>>                 //and the FunUtil.getNonEmptyMemberChildren() method for
>> .Children call*
>> *                if (exp != null &&
>> (exp.toString().indexOf("Descendants") > 0 ||
>> exp.toString().indexOf("Children")>0) )
>>                 {
>>                   slicerEvaluator.setNonEmpty(nonEmpty);
>>
>>                 }
>> *
>> It seems that the Descendants function with Mondrian 3.0.4 will only
>> return Non Empty member children, even if it is not specified inside of NON
>> EMPTY set. It will always join the fact table to only get the nonEmpty
>> member children. It behaves in the opposite way of the .Children call.
>>
>> with set [#DataSet#] as 'Descendants([Product].[All Products], 4)'
>> select {[Measures].[Store Invoice], [Measures].[Supply Time],
>> [Measures].[Warehouse Cost], [Measures].[Warehouse Sales]} ON COLUMNS,
>> {[#DataSet#]} ON ROWS
>> from [Warehouse]
>> where ([Time].[1997].[Q4].[12])
>>
>> If we apply the NonEmpty flag to
>> DescendantsFunDef.descendantsByDepth(Member, List, SchemaReader, int,
>> boolean, boolean, boolean, Evaluator) line: 163   like below. With this
>> change and the change above, The Descendants function will also return empty
>> member when it is not specified inside of NON EMPTY set.
>>
>>             if (context!= null && context.isNonEmpty())
>>             {
>>              children = schemaReader.getMemberChildren(children, context);
>>             }
>>             else
>>             {
>>              children = schemaReader.getMemberChildren(children);
>>             }
>>
>> Any suggestions or thoughts on this fix?
>>
>> Thanks,
>> Wenjin
>>
>>
>> On Wed, May 13, 2009 at 4:12 AM, Julian Hyde <jhyde at pentaho.com> wrote:
>>
>>> This sounds similar to http://jira.pentaho.com/browse/MONDRIAN-506 which
>>> you
>>> logged in February.
>>>
>>> It also sounds like the issue I wrote about in this blog post:
>>>
>>> http://julianhyde.blogspot.com/2009/01/hard-won-lessons-in-mondrian-query.ht
>>> ml
>>>
>>> If you read the blog post, you will see I had the same problem you did -
>>> every time I changed something to get the query to perform, something
>>> else
>>> broke.
>>>
>>> So, read the blog post. My conclusions there are the best I have to offer
>>> right now.
>>>
>>> Julian
>>>
>>> > -----Original Message-----
>>> > From: mondrian-bounces at pentaho.org
>>> > [mailto:mondrian-bounces at pentaho.org] On Behalf Of Robin Tharappel
>>> > Sent: Tuesday, May 12, 2009 12:49 PM
>>> > To: mondrian at pentaho.org
>>> > Subject: [Mondrian] .Children function not being evaluated
>>> > natively (no joinwith the fact table)
>>> >
>>> > Hello,
>>> >
>>> > With Mondrian 3.0.4 it appears that the .children function is not
>>> > being evaluated natively when specified inside of NON EMPTY set. It
>>> > will return empty children as well.  This can be a performance problem
>>> > if the dimension contains a large number of members at the children
>>> > level.  By joining the dimension table with the fact table the number
>>> > of members could be reduced . This is the MDX query with the
>>> > FoodMart.xml.
>>> >
>>> >
>>> > with set [#DataSet#] as '{[Product].[All Products].[Food].[Baking
>>> > Goods].[Baking Goods].Children}'
>>> > select {[Measures].[Store Invoice], [Measures].[Supply Time],
>>> > [Measures].[Warehouse Cost], [Measures].[Warehouse Sales]} ON COLUMNS,
>>> >  NON EMPTY Hierarchize({[#DataSet#]}) ON ROWS
>>> > from [Warehouse]
>>> > where ([Time].[1997].[Q4].[12])
>>> >
>>> >
>>> > The root cause of this problem is that for this expression
>>> > "[Product].[All Products].[Food].[Baking Goods].[Baking
>>> > Goods].Children", the RolapEvaluator's nonempty is true, but the
>>> > RolapEvaluator.root.slicerEvaluator's nonEmpty property is still
>>> > false. They are out of sync.
>>> >
>>> > The RolapEvaluator.evaluateNamedSet(String, Exp) method will call the
>>> > root.evaluateNamedSet(String, Exp). And it will pass the
>>> > root.slicerEvaluator to the
>>> > FunUtil.getNonEmptyMemberChildren(Evaluator, Member) method for the
>>> > .Children call, which will use this nonEmpty property of the
>>> > slicerEvalutor to determine weather to return the nonEmpty children or
>>> > not.
>>> >
>>> >
>>> > But if I made this change in
>>> > RolapResult$RolapResultEvaluatorRoot.evaluateNamedSet(String, Exp)
>>> > line: 1194  to sync the slicerEvaluator's nonEmpty with Evaluator's
>>> > nonEmpty value. It would break these three unit tests.
>>> >
>>> > testTopMetricWithThreeLevelHierarchy (mondrian.test.TopBottomTest
>>> > testTotalingWhenIgnoreUnrelatedDimensionsPropertyIsTrue(mondri
>>> > an.test.IgnoreUnrelatedDimensionsTest
>>> > testIndependentSlicerMemberNonNative(mondrian.rolap.NonEmptyTest
>>> >
>>> >
>>> > Note that if the .Children function is replaced by an equivalent
>>> > Descendants function it will be evaluated by joining with the fact
>>> > table. Any suggestion for this problem?
>>> >
>>> > Thanks
>>> >
>>> >
>>> >
>>> > The trace stack for the .Children call:
>>> > Thread [main] (Suspended)
>>> >             SqlMemberSource.chooseAggStar(MemberChildrenConstraint,
>>> > RolapMember) line: 581
>>> >             SqlMemberSource.makeChildMemberSql(RolapMember,
>>> > DataSource, MemberChildrenConstraint) line: 502
>>> >             SqlMemberSource.getMemberChildren2(RolapMember,
>>> > List<RolapMember>, MemberChildrenConstraint) line: 836
>>> >             SqlMemberSource.getMemberChildren(RolapMember,
>>> > List<RolapMember>, MemberChildrenConstraint) line: 770
>>> >             SqlMemberSource.getMemberChildren(List<RolapMember>,
>>> > List<RolapMember>, MemberChildrenConstraint) line: 745
>>> >             SmartMemberReader.readMemberChildren(List<RolapMember>,
>>> > List<RolapMember>, MemberChildrenConstraint) line: 237
>>> >             SmartMemberReader.getMemberChildren(List<RolapMember>,
>>> > List<RolapMember>, MemberChildrenConstraint) line: 201
>>> >
>>> > RolapCubeHierarchy$RolapCubeHierarchyMemberReader.readMemberCh
>>> ildren(List<RolapMember>,
>>> > List<RolapMember>, MemberChildrenConstraint) line: 472
>>> >
>>> > RolapCubeHierarchy$RolapCubeHierarchyMemberReader.getMemberChi
>>> ldren(List<RolapMember>,
>>> > List<RolapMember>, MemberChildrenConstraint) line: 568
>>> >
>>> > RolapCubeHierarchy$RolapCubeHierarchyMemberReader(SmartMemberR
>>> eader).getMemberChildren(RolapMember,
>>> > List<RolapMember>, MemberChildrenConstraint) line: 169
>>> >
>>> > RolapCube$RolapCubeSchemaReader(RolapSchemaReader).internalGet
>>> MemberChildren(Member,
>>> > MemberChildrenConstraint) line: 155     //Here is
>>> > DefaultMemberChildrenConstraint
>>> >
>>> > RolapCube$RolapCubeSchemaReader(RolapSchemaReader).getMemberCh
>>> > ildren(Member,
>>> > Evaluator) line: 145       //Here the Evaluator is null
>>> >
>>> > RolapCube$RolapCubeSchemaReader(RolapSchemaReader).getMemberCh
>>> > ildren(Member)
>>> > line: 139
>>> >
>>> > Query$QuerySchemaReader(DelegatingSchemaReader).getMemberChild
>>> > ren(Member)
>>> > line: 60
>>> >             FunUtil.getNonEmptyMemberChildren(Evaluator, Member) line:
>>> > 1790
>>> >             BuiltinFunTable$30$1.evaluateList(Evaluator) line: 906
>>> >             SetFunDef$ListSetCalc$1.evaluateVoid(Evaluator) line: 131
>>> >             SetFunDef$ListSetCalc.evaluateList(Evaluator) line: 204
>>> >
>>> > SetFunDef$ListSetCalc(AbstractListCalc).evaluate(Evaluator)
>>> > line: 67
>>> >             RolapResult.evaluateExp(Calc, RolapEvaluator) line: 794
>>> >             RolapResult.access$100(RolapResult, Calc, RolapEvaluator)
>>> > line: 46
>>> >
>>> > RolapResult$RolapResultEvaluatorRoot.evaluateNamedSet(String,
>>> > Exp) line: 1194
>>> >             RolapEvaluator.evaluateNamedSet(String, Exp) line: 884
>>> >             NamedSetExpr$1.evaluateList(Evaluator) line: 78
>>> >             SetFunDef$ListSetCalc$1.evaluateVoid(Evaluator) line: 131
>>> >             SetFunDef$ListSetCalc.evaluateList(Evaluator) line: 204
>>> >             HierarchizeFunDef$1.evaluateList(Evaluator) line: 48
>>> >
>>> > HierarchizeFunDef$1(AbstractListCalc).evaluate(Evaluator) line: 67
>>> >             RolapResult.executeAxis(Evaluator, QueryAxis, Calc,
>>> > boolean, RolapResult$AxisMember) line: 694
>>> >             RolapResult.evalLoad(List<Member[]>, int, Evaluator,
>>> > QueryAxis, Calc, AxisMember) line: 557
>>> >             RolapResult.loadMembers(List<Member[]>, RolapEvaluator,
>>> > QueryAxis, Calc, AxisMember) line: 532
>>> >             RolapResult.<init>(Query, boolean) line: 254
>>> >             RolapConnection.execute(Query) line: 597
>>> >             CmdRunner.runQuery(String, boolean) line: 566
>>> >             CmdRunner.execute(String) line: 543
>>> >             CmdRunner.executeMdxCmd(String) line: 2243
>>> >             CmdRunner.commandLoop(Reader, boolean) line: 872
>>> >             CmdRunner.commandLoop(File) line: 750
>>> >             CmdRunner.main(String[]) line: 2397
>>> >
>>> >    FunUtil.java
>>> >
>>> >   public static Member[] getNonEmptyMemberChildren(
>>> >         Evaluator evaluator,
>>> >         Member member)
>>> >     {
>>> >         SchemaReader sr = evaluator.getSchemaReader();
>>> >         if (evaluator.isNonEmpty()) {
>>> >             return sr.getMemberChildren(member, evaluator);
>>> >         } else {
>>> >             return sr.getMemberChildren(member);
>>> >         }
>>> >     }
>>> >
>>> > _______________________________________________
>>> > Mondrian mailing list
>>> > Mondrian at pentaho.org
>>> > http://lists.pentaho.org/mailman/listinfo/mondrian
>>> >
>>> >
>>> >
>>>
>>>
>>> _______________________________________________
>>> Mondrian mailing list
>>> Mondrian at pentaho.org
>>> http://lists.pentaho.org/mailman/listinfo/mondrian
>>>
>>
>>
>> _______________________________________________
>> Mondrian mailing list
>> Mondrian at pentaho.org
>> http://lists.pentaho.org/mailman/listinfo/mondrian
>>
>>
>
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20090527/fa21fb57/attachment.html 


More information about the Mondrian mailing list