[Mondrian] RE: 3.2 NonEmptyCrossJoin Issues

Julian Hyde jhyde at pentaho.com
Mon Apr 26 23:32:06 EDT 2010 

Will,

It looks very reasonable. I am running your change through my suite of tests
now.

I think you should check in before the 12 midnight EDT mondrian regression
suite starts. If my suite shows up a problem, I'll let you know and we can
back out.

Matt/Kurtis,

Can you please comment because it was your change 13151 that added 

         if (evaluator.nativeEnabled())

into RolapSchemaReader.getNativeSetEvaluator.

Julian

> -----Original Message-----
> From: Will Gorman [mailto:wgorman at pentaho.com] 
> Sent: Monday, April 26, 2010 8:16 PM
> To: Julian Hyde
> Cc: Benny Chow; Jake Cornelius
> Subject: RE: 3.2 NonEmptyCrossJoin Issues
> 
> Hi Julian,
> 
> I've narrowed the change down to
> RolapSchemaReader.getNativeSetEvaluator(), it is now checking for
> evaluator.enableNative() before creating a native evaluator.  
> I was able
> to get the unit test to pass by changing the default value of
> RolapEvaluator.enableNative in the constructor from:
> 	MondrianProperties.instance().EnableNativeNonEmpty.get()
> to:
> 	MondrianProperties.instance().EnableNativeNonEmpty.get() ||
> 	MondrianProperties.instance().EnableNativeCrossJoin.get(); 
> 
> All the unit tests pass.  Does that seem like a reasonable 
> approach?  If
> I just remove the check in RolapSchemaReader there are test failures.
> 
> Thanks,
> 
> Will
> 
> On Mon, 2010-04-26 at 22:15 -0400, Will Gorman wrote:
> > After running a quick binary search through change lists, 
> I've narrowed
> > down the regression to change #13151 - "MONDRIAN - 
> NativizeSet - making
> > sure each high cardinality determination is not affected by the
> > evaluation of previous axes."
> > 
> > Also, I don't think the slimmed down unit test is really 
> testing for the
> > issue, the exception that is thrown is in a different area 
> of the code.
> > 
> > Will
> > 
> > On Mon, 2010-04-26 at 17:52 -0400, Will Gorman wrote:
> > > Hi Julian,
> > > 
> > > Attached are two unit test cases that demonstrate the 
> issue.  I used the
> > > original MDX and Benny's slimmed down version for the two 
> test cases.
> > > Hopefully this helps!
> > > 
> > > Will
> > > 
> > > On Sat, 2010-04-24 at 04:59 -0500, Benny Chow wrote:
> > > > Julian,
> > > > 
> > > > I *think* the regression was caused by your change 
> 13234 on 12/16/2009
> > > > 03:02:56 AM MONDRIAN: Rationalize & tune how root evaluator is
> > > > created.
> > > > 
> > > > Mondrian's code is complicated and I'm not very clear 
> on what's going
> > > > on in there.  But looking at the changes in RolapEvaluator I'm
> > > > guessing that when the properties were set to:
> > > > 
> > > > mondrian.native.crossjoin.enable=true 
> > > > mondrian.native.nonempty.enable=false
> > > > 
> > > > Native CJ evaluation was still happening but after 
> 13234, native CJ
> > > > stopped kicking in.
> > > > 
> > > > Here's a simple query that should be native evaluated 
> with the above
> > > > property settings:
> > > > 
> > > > With 
> > > >  Set [*NATIVE_CJ_SET] as 'NonEmptyCrossJoin([Education
> > > > Level].[Education Level].Members,[Store Type].[Store 
> Type].Members)'
> > > > 
> > > >  Select 
> > > >  [*NATIVE_CJ_SET]  on columns 
> > > >  From [Sales]
> > > > 
> > > > Is this enough information for you to try to fix this?  
> > > > 
> > > > Benny
> > > > 
> > > > 
> > > > 
> > > > -----Original Message----- 
> > > > From: Benny Chow 
> > > > Sent: Sat 4/24/2010 12:24 PM 
> > > > To: Will Gorman 
> > > > Cc: Julian Hyde; Jake Cornelius 
> > > > Subject: RE: 3.2 NonEmptyCrossJoin Issues 
> > > >   
> > > > Hi Will,
> > > > 
> > > > I checked in the code.  There are currently three test 
> failures.  2
> > > > are related to Marc's change which I sent email earlier 
> about.  1 is
> > > > related to the nonemptycrossjoin issue.  
> > > > 
> > > > IMO, it's a blocker since it's a big performance issue. 
>  Analyzer MDX
> > > > generation was architected to use the NECJ to return only valid
> > > > combinations of attributes on the report.
> > > > 
> > > > I'll do some more investigation over the weekend.
> > > > 
> > > > Benny
> > > > 
> > > > 
> > > > -----Original Message----- 
> > > > From: Will Gorman 
> > > > Sent: Saturday, April 24, 2010 5:30 AM 
> > > > To: Benny Chow 
> > > > Cc: Julian Hyde; Jake Cornelius 
> > > > Subject: Re: 3.2 NonEmptyCrossJoin Issues
> > > > 
> > > > Hi Benny,
> > > > 
> > > > Thanks for doing the research.  Could you check in your 
> progress?
> > > > I've 
> > > > made similar changes on the reporting side, and 
> verified all JPivot
> > > > unit 
> > > > tests are passing.  
> > > > 
> > > > Also, do you feel that the native regression is a 
> blocker for our RC 
> > > > release, or are you okay with waiting till GA to have 
> this resolved?
> > > > 
> > > > Thanks!
> > > > 
> > > > Will
> > > > 
> > > > On Fri, 2010-04-23 at 12:06 -0500, Benny Chow wrote: 
> > > > > Julian, 
> > > > > 
> > > > > I refactored Analyzer's code to not depend on the All member
> > > > anymore 
> > > > > and use getParentMember to find parent members.  The 
> majority of 
> > > > > Analyzer's unit tests pass now.  I think next time 
> you fix old 
> > > > > LucidEra Clearview unit tests, you should give me a 
> heads up since
> > > > all 
> > > > > of those tests exist in Analyzer.  
> > > > > 
> > > > > The NonEmptyCrossJoin issue I am running into is that native 
> > > > > evaluation is not happening anymore for the below query: 
> > > > > 
> > > > > 
> > > > > With 
> > > > > Set [*NATIVE_CJ_SET] as 
> 'NonEmptyCrossJoin([*BASE_MEMBERS_Education 
> > > > > Level],NonEmptyCrossJoin([*BASE_MEMBERS_Common 
> Date-Canonical Dates 
> > > > > Calendar Hierarchy],[*BASE_MEMBERS_Product]))' 
> > > > > 
> > > > > Set [*BASE_MEMBERS_Common Date-Canonical Dates 
> Calendar Hierarchy]
> > > > as 
> > > > > '[Common Date-Canonical Dates Calendar Hierarchy].[Calendar 
> > > > > Month].Members' 
> > > > > 
> > > > > Set [*BASE_MEMBERS_Education Level] as '[Education
> > > > Level].[Education 
> > > > > Level].Members' 
> > > > > Set [*BASE_MEMBERS_Product] as '[Product].[Product 
> Family].Members' 
> > > > > Set [*BASE_MEMBERS_Measures] as
> > > > '{[Measures].[*FORMATTED_MEASURE_0]}' 
> > > > > Member [Measures].[*FORMATTED_MEASURE_0] as 
> '[Measures].[Sales 
> > > > > Count]', FORMAT_STRING = '#,###', SOLVE_ORDER=400 
> > > > > Select 
> > > > > [*BASE_MEMBERS_Measures] on columns, 
> > > > > Non Empty [*NATIVE_CJ_SET] on rows 
> > > > > From [Sales] 
> > > > > 
> > > > > 
> > > > > The [*NATIVE_CJ_SET] should have been natively 
> evaluated with all 
> > > > > three dimensions pushed down to the DB.  
> > > > > 
> > > > > The current mondrian.properties is the same as I sent you
> > > > previously. 
> > > > > I think the two key properties are: 
> > > > > 
> > > > > # if enabled some NON EMPTY CrossJoin will be computed in SQL 
> > > > > 
> > > > > mondrian.native.crossjoin.enable=true 
> > > > > 
> > > > > # some NON EMPTY set operations like member.children, 
> level.members 
> > > > > and member descendants will be computed in SQL 
> > > > > 
> > > > > mondrian.native.nonempty.enable=false 
> > > > > 
> > > > > 
> > > > > These properties have not changed so the behavior 
> should not have 
> > > > > changed either.  
> > > > > 
> > > > > Analyzer has a bunch of unit tests that test for 
> native evaulation
> > > > by 
> > > > > turning on 
> > > > > 
> MondrianProperties.instance().AlertNativeEvaluationUnsupported.
> > > > (This 
> > > > > was something JVS added). But because of this 
> regression, Mondrian
> > > > is 
> > > > > not reporting when native evaluation is not being used (but
> > > > expected). 
> > > > > 
> > > > > I'm still investigating this but if you know that 
> code has been 
> > > > > changed in this area you can give me some pointers... 
> > > > > 
> > > > > Thanks 
> > > > > Benny 
> > > > > 
> > > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > 
> 
> 
>

More information about the Mondrian mailing list