[Mondrian] RE: 3.2 NonEmptyCrossJoin Issues

Kurtis.Walker at thomsonreuters.com Kurtis.Walker at thomsonreuters.com
Tue Apr 27 09:06:44 EDT 2010


Will/Julian,

I'm curious what it means to turn off NativeNonEmpty but turn on
NativeCrossjoin.  I usually think of NativeNonEmpty as the "parent"
property, meaning if it is off, then none of the child properties
(NativeCrossjoin, NativeTopCount, etc) have any effect.  Is this the
wrong way to think about it?  If NativeCrossjoin should turn on native
regardless of the NativeNonEmpty value, then the proposed change makes
sense to me.  Also, can you send me which test cases are failing?  They
were attached to one of the previous e-mails in this chain.  Thanks.

Kurt

-----Original Message-----
From: Julian Hyde [mailto:jhyde at pentaho.com] 
Sent: Monday, April 26, 2010 11:32 PM
To: 'Will Gorman'; Campbell, Matthew (Hlthcr&Science); Walker, Kurtis
(Hlthcr&Science)
Cc: 'Benny Chow'; 'Jake Cornelius'; 'Mondrian developer mailing list'
Subject: RE: 3.2 NonEmptyCrossJoin Issues

Will,

It looks very reasonable. I am running your change through my suite of
tests
now.

I think you should check in before the 12 midnight EDT mondrian
regression
suite starts. If my suite shows up a problem, I'll let you know and we
can
back out.

Matt/Kurtis,

Can you please comment because it was your change 13151 that added 

         if (evaluator.nativeEnabled())

into RolapSchemaReader.getNativeSetEvaluator.

Julian

> -----Original Message-----
> From: Will Gorman [mailto:wgorman at pentaho.com] 
> Sent: Monday, April 26, 2010 8:16 PM
> To: Julian Hyde
> Cc: Benny Chow; Jake Cornelius
> Subject: RE: 3.2 NonEmptyCrossJoin Issues
> 
> Hi Julian,
> 
> I've narrowed the change down to
> RolapSchemaReader.getNativeSetEvaluator(), it is now checking for
> evaluator.enableNative() before creating a native evaluator.  
> I was able
> to get the unit test to pass by changing the default value of
> RolapEvaluator.enableNative in the constructor from:
> 	MondrianProperties.instance().EnableNativeNonEmpty.get()
> to:
> 	MondrianProperties.instance().EnableNativeNonEmpty.get() ||
> 	MondrianProperties.instance().EnableNativeCrossJoin.get(); 
> 
> All the unit tests pass.  Does that seem like a reasonable 
> approach?  If
> I just remove the check in RolapSchemaReader there are test failures.
> 
> Thanks,
> 
> Will
> 
> On Mon, 2010-04-26 at 22:15 -0400, Will Gorman wrote:
> > After running a quick binary search through change lists, 
> I've narrowed
> > down the regression to change #13151 - "MONDRIAN - 
> NativizeSet - making
> > sure each high cardinality determination is not affected by the
> > evaluation of previous axes."
> > 
> > Also, I don't think the slimmed down unit test is really 
> testing for the
> > issue, the exception that is thrown is in a different area 
> of the code.
> > 
> > Will
> > 
> > On Mon, 2010-04-26 at 17:52 -0400, Will Gorman wrote:
> > > Hi Julian,
> > > 
> > > Attached are two unit test cases that demonstrate the 
> issue.  I used the
> > > original MDX and Benny's slimmed down version for the two 
> test cases.
> > > Hopefully this helps!
> > > 
> > > Will
> > > 
> > > On Sat, 2010-04-24 at 04:59 -0500, Benny Chow wrote:
> > > > Julian,
> > > > 
> > > > I *think* the regression was caused by your change 
> 13234 on 12/16/2009
> > > > 03:02:56 AM MONDRIAN: Rationalize & tune how root evaluator is
> > > > created.
> > > > 
> > > > Mondrian's code is complicated and I'm not very clear 
> on what's going
> > > > on in there.  But looking at the changes in RolapEvaluator I'm
> > > > guessing that when the properties were set to:
> > > > 
> > > > mondrian.native.crossjoin.enable=true 
> > > > mondrian.native.nonempty.enable=false
> > > > 
> > > > Native CJ evaluation was still happening but after 
> 13234, native CJ
> > > > stopped kicking in.
> > > > 
> > > > Here's a simple query that should be native evaluated 
> with the above
> > > > property settings:
> > > > 
> > > > With 
> > > >  Set [*NATIVE_CJ_SET] as 'NonEmptyCrossJoin([Education
> > > > Level].[Education Level].Members,[Store Type].[Store 
> Type].Members)'
> > > > 
> > > >  Select 
> > > >  [*NATIVE_CJ_SET]  on columns 
> > > >  From [Sales]
> > > > 
> > > > Is this enough information for you to try to fix this?  
> > > > 
> > > > Benny
> > > > 
> > > > 
> > > > 
> > > > -----Original Message----- 
> > > > From: Benny Chow 
> > > > Sent: Sat 4/24/2010 12:24 PM 
> > > > To: Will Gorman 
> > > > Cc: Julian Hyde; Jake Cornelius 
> > > > Subject: RE: 3.2 NonEmptyCrossJoin Issues 
> > > >   
> > > > Hi Will,
> > > > 
> > > > I checked in the code.  There are currently three test 
> failures.  2
> > > > are related to Marc's change which I sent email earlier 
> about.  1 is
> > > > related to the nonemptycrossjoin issue.  
> > > > 
> > > > IMO, it's a blocker since it's a big performance issue. 
>  Analyzer MDX
> > > > generation was architected to use the NECJ to return only valid
> > > > combinations of attributes on the report.
> > > > 
> > > > I'll do some more investigation over the weekend.
> > > > 
> > > > Benny
> > > > 
> > > > 
> > > > -----Original Message----- 
> > > > From: Will Gorman 
> > > > Sent: Saturday, April 24, 2010 5:30 AM 
> > > > To: Benny Chow 
> > > > Cc: Julian Hyde; Jake Cornelius 
> > > > Subject: Re: 3.2 NonEmptyCrossJoin Issues
> > > > 
> > > > Hi Benny,
> > > > 
> > > > Thanks for doing the research.  Could you check in your 
> progress?
> > > > I've 
> > > > made similar changes on the reporting side, and 
> verified all JPivot
> > > > unit 
> > > > tests are passing.  
> > > > 
> > > > Also, do you feel that the native regression is a 
> blocker for our RC 
> > > > release, or are you okay with waiting till GA to have 
> this resolved?
> > > > 
> > > > Thanks!
> > > > 
> > > > Will
> > > > 
> > > > On Fri, 2010-04-23 at 12:06 -0500, Benny Chow wrote: 
> > > > > Julian, 
> > > > > 
> > > > > I refactored Analyzer's code to not depend on the All member
> > > > anymore 
> > > > > and use getParentMember to find parent members.  The 
> majority of 
> > > > > Analyzer's unit tests pass now.  I think next time 
> you fix old 
> > > > > LucidEra Clearview unit tests, you should give me a 
> heads up since
> > > > all 
> > > > > of those tests exist in Analyzer.  
> > > > > 
> > > > > The NonEmptyCrossJoin issue I am running into is that native 
> > > > > evaluation is not happening anymore for the below query: 
> > > > > 
> > > > > 
> > > > > With 
> > > > > Set [*NATIVE_CJ_SET] as 
> 'NonEmptyCrossJoin([*BASE_MEMBERS_Education 
> > > > > Level],NonEmptyCrossJoin([*BASE_MEMBERS_Common 
> Date-Canonical Dates 
> > > > > Calendar Hierarchy],[*BASE_MEMBERS_Product]))' 
> > > > > 
> > > > > Set [*BASE_MEMBERS_Common Date-Canonical Dates 
> Calendar Hierarchy]
> > > > as 
> > > > > '[Common Date-Canonical Dates Calendar Hierarchy].[Calendar 
> > > > > Month].Members' 
> > > > > 
> > > > > Set [*BASE_MEMBERS_Education Level] as '[Education
> > > > Level].[Education 
> > > > > Level].Members' 
> > > > > Set [*BASE_MEMBERS_Product] as '[Product].[Product 
> Family].Members' 
> > > > > Set [*BASE_MEMBERS_Measures] as
> > > > '{[Measures].[*FORMATTED_MEASURE_0]}' 
> > > > > Member [Measures].[*FORMATTED_MEASURE_0] as 
> '[Measures].[Sales 
> > > > > Count]', FORMAT_STRING = '#,###', SOLVE_ORDER=400 
> > > > > Select 
> > > > > [*BASE_MEMBERS_Measures] on columns, 
> > > > > Non Empty [*NATIVE_CJ_SET] on rows 
> > > > > From [Sales] 
> > > > > 
> > > > > 
> > > > > The [*NATIVE_CJ_SET] should have been natively 
> evaluated with all 
> > > > > three dimensions pushed down to the DB.  
> > > > > 
> > > > > The current mondrian.properties is the same as I sent you
> > > > previously. 
> > > > > I think the two key properties are: 
> > > > > 
> > > > > # if enabled some NON EMPTY CrossJoin will be computed in SQL 
> > > > > 
> > > > > mondrian.native.crossjoin.enable=true 
> > > > > 
> > > > > # some NON EMPTY set operations like member.children, 
> level.members 
> > > > > and member descendants will be computed in SQL 
> > > > > 
> > > > > mondrian.native.nonempty.enable=false 
> > > > > 
> > > > > 
> > > > > These properties have not changed so the behavior 
> should not have 
> > > > > changed either.  
> > > > > 
> > > > > Analyzer has a bunch of unit tests that test for 
> native evaulation
> > > > by 
> > > > > turning on 
> > > > > 
> MondrianProperties.instance().AlertNativeEvaluationUnsupported.
> > > > (This 
> > > > > was something JVS added). But because of this 
> regression, Mondrian
> > > > is 
> > > > > not reporting when native evaluation is not being used (but
> > > > expected). 
> > > > > 
> > > > > I'm still investigating this but if you know that 
> code has been 
> > > > > changed in this area you can give me some pointers... 
> > > > > 
> > > > > Thanks 
> > > > > Benny 
> > > > > 
> > > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > 
> 
> 
> 





More information about the Mondrian mailing list