[Mondrian] .Children function not being evaluated natively (no joinwith the fact table)

Julian Hyde jhyde at pentaho.com
Fri May 15 15:03:58 EDT 2009


Wenjin,
 
I'm nervous about accepting your change, because I don't understand the
current code and this makes it yet more complicated. As Matt says, this is a
general problem, not just related to Children and Descendants, so any fix
that looks for specific function names is probably wrong. I want to change
this code - as described in my blog post - and extra complications will make
it more difficult to refactor later.
 
So, I'd like you to create a unit test for this problem and contribute that.
I saw you alrady have a query against FoodMart  that reproduces the problem.
Add one or two methods to NonEmptyTest. Also log a bug, and make the test
cases dependent on 'if (Bug.BugMondrianXXXFixed) ...'. Then test against 3.1
to see whether the bug still exists.
 
Post a patch that fixes your test cases to this list, and I will consider
integrating it. Even if I don't accept the patch, your test case will give
us something to aim for when we refactor to solve this problem the 'right'
way.
 
Julian


  _____  

From: mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org] On
Behalf Of Matt Campbell
Sent: Friday, May 15, 2009 7:40 AM
To: Mondrian developer mailing list
Subject: Re: [Mondrian] .Children function not being evaluated natively (no
joinwith the fact table)


The bug you identified when using the Descendants() function is more general
than just in named sets, and is probably more general than just using
Descendants().   

SqlConstraintFactory will return a SqlContextConstraint, even if the current
context is not nonempty.  Executing the query you posted will result in a
SCC being used when evaluating the tuple list, which causes native non empty
behavior.  I think a SqlContextConstraint should only be used in a non empty
context.

For other MDX functions (like .Children and .Members) there is an upstream
check for nonempty, which causes a null context to be passed to
SqlConstraintFactory.  If SqlConstraintFactory finds a null context it will
return a DefaultMemberContextConstraint, resulting in the desired set.


I think the simplest solution here would be to add a check for non empty
context in the SqlConstraintFactory methods.


On Thu, May 14, 2009 at 8:04 PM, wenjin huang <wenjin.huang at gmail.com>
wrote:


Hi, 
 
Thank you for your reply.
 
We did some testing to sync the nonEmpty flag for root.slicerEvaluator with
its parent/caller RoalEvalutor. We tried the change below in
RolapResultEvaluatorRoot.evaluateNamedSet() method. With this change, The
.Children call will only return Non Empty children when it is specified
inside of NON EMPTY set. Since this change only applies to .Children and
Descendants function, it doesn't break any unit tests so far.
 
RolapResult$RolapResultEvaluatorRoot.evaluateNamedSet(String, Exp) line:
1194  

        //Pass in nonEmpty flag from the caller/parent RoalEvalutor
        protected Object evaluateNamedSet(String name, Exp exp, boolean
nonEmpty) {
            Object value = namedSetValues.get(name);
            if (value == null) {
                final RolapEvaluator.RolapEvaluatorRoot root =
                    slicerEvaluator.root;
                final Calc calc = root.getCompiled(exp, false,
ResultStyle.LIST);
                
                //Sync the nonEmpty flag for root.slicerEvaluator with its
parent/caller RoalEvalutor
                //Restrict the change only to .Children call and Descendants
function.
                //This flag will be used in the
DescendantsFunDef.descendantsByDepth() method for Descendants function,
                //and the FunUtil.getNonEmptyMemberChildren() method for
.Children call
                if (exp != null && (exp.toString().indexOf("Descendants") >
0 || exp.toString().indexOf("Children")>0) )
                {
                  slicerEvaluator.setNonEmpty(nonEmpty);
                  
                }

It seems that the Descendants function with Mondrian 3.0.4 will only return
Non Empty member children, even if it is not specified inside of NON EMPTY
set. It will always join the fact table to only get the nonEmpty member
children. It behaves in the opposite way of the .Children call. 
 
with set [#DataSet#] as 'Descendants([Product].[All Products], 4)' 

select {[Measures].[Store Invoice], [Measures].[Supply Time],
[Measures].[Warehouse Cost], [Measures].[Warehouse Sales]} ON COLUMNS,

{[#DataSet#]} ON ROWS
from [Warehouse]
where ([Time].[1997].[Q4].[12])

If we apply the NonEmpty flag to
DescendantsFunDef.descendantsByDepth(Member, List, SchemaReader, int,
boolean, boolean, boolean, Evaluator) line: 163   like below. With this
change and the change above, The Descendants function will also return empty
member when it is not specified inside of NON EMPTY set.
 
            if (context!= null && context.isNonEmpty())
            {
             children = schemaReader.getMemberChildren(children, context);
            }
            else
            {
             children = schemaReader.getMemberChildren(children);
            }
   
Any suggestions or thoughts on this fix? 
 
Thanks, 
Wenjin
 
 
On Wed, May 13, 2009 at 4:12 AM, Julian Hyde <jhyde at pentaho.com> wrote:


This sounds similar to http://jira.pentaho.com/browse/MONDRIAN-506 which you
logged in February.

It also sounds like the issue I wrote about in this blog post:
http://julianhyde.blogspot.com/2009/01/hard-won-lessons-in-mondrian-query.ht
<http://julianhyde.blogspot.com/2009/01/hard-won-lessons-in-mondrian-query.h
tml> 
ml

If you read the blog post, you will see I had the same problem you did -
every time I changed something to get the query to perform, something else
broke.

So, read the blog post. My conclusions there are the best I have to offer
right now.

Julian

> -----Original Message-----
> From: mondrian-bounces at pentaho.org
> [mailto:mondrian-bounces at pentaho.org] On Behalf Of Robin Tharappel
> Sent: Tuesday, May 12, 2009 12:49 PM
> To: mondrian at pentaho.org
> Subject: [Mondrian] .Children function not being evaluated
> natively (no joinwith the fact table)
>
> Hello,
>
> With Mondrian 3.0.4 it appears that the .children function is not
> being evaluated natively when specified inside of NON EMPTY set. It
> will return empty children as well.  This can be a performance problem
> if the dimension contains a large number of members at the children
> level.  By joining the dimension table with the fact table the number
> of members could be reduced . This is the MDX query with the
> FoodMart.xml.
>
>
> with set [#DataSet#] as '{[Product].[All Products].[Food].[Baking
> Goods].[Baking Goods].Children}'
> select {[Measures].[Store Invoice], [Measures].[Supply Time],
> [Measures].[Warehouse Cost], [Measures].[Warehouse Sales]} ON COLUMNS,
>  NON EMPTY Hierarchize({[#DataSet#]}) ON ROWS
> from [Warehouse]
> where ([Time].[1997].[Q4].[12])
>
>
> The root cause of this problem is that for this expression
> "[Product].[All Products].[Food].[Baking Goods].[Baking
> Goods].Children", the RolapEvaluator's nonempty is true, but the
> RolapEvaluator.root.slicerEvaluator's nonEmpty property is still
> false. They are out of sync.
>
> The RolapEvaluator.evaluateNamedSet(String, Exp) method will call the
> root.evaluateNamedSet(String, Exp). And it will pass the
> root.slicerEvaluator to the
> FunUtil.getNonEmptyMemberChildren(Evaluator, Member) method for the
> .Children call, which will use this nonEmpty property of the
> slicerEvalutor to determine weather to return the nonEmpty children or
> not.
>
>
> But if I made this change in
> RolapResult$RolapResultEvaluatorRoot.evaluateNamedSet(String, Exp)
> line: 1194  to sync the slicerEvaluator's nonEmpty with Evaluator's
> nonEmpty value. It would break these three unit tests.
>
> testTopMetricWithThreeLevelHierarchy (mondrian.test.TopBottomTest
> testTotalingWhenIgnoreUnrelatedDimensionsPropertyIsTrue(mondri
> an.test.IgnoreUnrelatedDimensionsTest
> testIndependentSlicerMemberNonNative(mondrian.rolap.NonEmptyTest
>
>
> Note that if the .Children function is replaced by an equivalent
> Descendants function it will be evaluated by joining with the fact
> table. Any suggestion for this problem?
>
> Thanks
>
>
>
> The trace stack for the .Children call:
> Thread [main] (Suspended)
>             SqlMemberSource.chooseAggStar(MemberChildrenConstraint,
> RolapMember) line: 581
>             SqlMemberSource.makeChildMemberSql(RolapMember,
> DataSource, MemberChildrenConstraint) line: 502
>             SqlMemberSource.getMemberChildren2(RolapMember,
> List<RolapMember>, MemberChildrenConstraint) line: 836
>             SqlMemberSource.getMemberChildren(RolapMember,
> List<RolapMember>, MemberChildrenConstraint) line: 770
>             SqlMemberSource.getMemberChildren(List<RolapMember>,
> List<RolapMember>, MemberChildrenConstraint) line: 745
>             SmartMemberReader.readMemberChildren(List<RolapMember>,
> List<RolapMember>, MemberChildrenConstraint) line: 237
>             SmartMemberReader.getMemberChildren(List<RolapMember>,
> List<RolapMember>, MemberChildrenConstraint) line: 201
>
> RolapCubeHierarchy$RolapCubeHierarchyMemberReader.readMemberCh
ildren(List<RolapMember>,
> List<RolapMember>, MemberChildrenConstraint) line: 472
>
> RolapCubeHierarchy$RolapCubeHierarchyMemberReader.getMemberChi
ldren(List<RolapMember>,
> List<RolapMember>, MemberChildrenConstraint) line: 568
>
> RolapCubeHierarchy$RolapCubeHierarchyMemberReader(SmartMemberR
eader).getMemberChildren(RolapMember,
> List<RolapMember>, MemberChildrenConstraint) line: 169
>
> RolapCube$RolapCubeSchemaReader(RolapSchemaReader).internalGet
MemberChildren(Member,
> MemberChildrenConstraint) line: 155     //Here is
> DefaultMemberChildrenConstraint
>
> RolapCube$RolapCubeSchemaReader(RolapSchemaReader).getMemberCh
> ildren(Member,
> Evaluator) line: 145       //Here the Evaluator is null
>
> RolapCube$RolapCubeSchemaReader(RolapSchemaReader).getMemberCh
> ildren(Member)
> line: 139
>
> Query$QuerySchemaReader(DelegatingSchemaReader).getMemberChild
> ren(Member)
> line: 60
>             FunUtil.getNonEmptyMemberChildren(Evaluator, Member) line:
> 1790
>             BuiltinFunTable$30$1.evaluateList(Evaluator) line: 906
>             SetFunDef$ListSetCalc$1.evaluateVoid(Evaluator) line: 131
>             SetFunDef$ListSetCalc.evaluateList(Evaluator) line: 204
>
> SetFunDef$ListSetCalc(AbstractListCalc).evaluate(Evaluator)
> line: 67
>             RolapResult.evaluateExp(Calc, RolapEvaluator) line: 794
>             RolapResult.access$100(RolapResult, Calc, RolapEvaluator)
> line: 46
>
> RolapResult$RolapResultEvaluatorRoot.evaluateNamedSet(String,
> Exp) line: 1194
>             RolapEvaluator.evaluateNamedSet(String, Exp) line: 884
>             NamedSetExpr$1.evaluateList(Evaluator) line: 78
>             SetFunDef$ListSetCalc$1.evaluateVoid(Evaluator) line: 131
>             SetFunDef$ListSetCalc.evaluateList(Evaluator) line: 204
>             HierarchizeFunDef$1.evaluateList(Evaluator) line: 48
>
> HierarchizeFunDef$1(AbstractListCalc).evaluate(Evaluator) line: 67
>             RolapResult.executeAxis(Evaluator, QueryAxis, Calc,
> boolean, RolapResult$AxisMember) line: 694
>             RolapResult.evalLoad(List<Member[]>, int, Evaluator,
> QueryAxis, Calc, AxisMember) line: 557
>             RolapResult.loadMembers(List<Member[]>, RolapEvaluator,
> QueryAxis, Calc, AxisMember) line: 532
>             RolapResult.<init>(Query, boolean) line: 254
>             RolapConnection.execute(Query) line: 597
>             CmdRunner.runQuery(String, boolean) line: 566
>             CmdRunner.execute(String) line: 543
>             CmdRunner.executeMdxCmd(String) line: 2243
>             CmdRunner.commandLoop(Reader, boolean) line: 872
>             CmdRunner.commandLoop(File) line: 750
>             CmdRunner.main(String[]) line: 2397
>
>    FunUtil.java
>
>   public static Member[] getNonEmptyMemberChildren(
>         Evaluator evaluator,
>         Member member)
>     {
>         SchemaReader sr = evaluator.getSchemaReader();
>         if (evaluator.isNonEmpty()) {
>             return sr.getMemberChildren(member, evaluator);
>         } else {
>             return sr.getMemberChildren(member);
>         }
>     }
>
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>
>
>


_______________________________________________
Mondrian mailing list
Mondrian at pentaho.org
http://lists.pentaho.org/mailman/listinfo/mondrian




_______________________________________________
Mondrian mailing list
Mondrian at pentaho.org
http://lists.pentaho.org/mailman/listinfo/mondrian




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20090515/8763373a/attachment.html 


More information about the Mondrian mailing list