[Mondrian] Re: 3.2 NonEmptyCrossJoin Issues

Will Gorman wgorman at pentaho.com
Tue Apr 27 09:29:17 EDT 2010


I checked in a test case with the change.

Thanks!

Will

On Apr 27, 2010, at 9:06 AM, "Kurtis.Walker at thomsonreuters.com" <Kurtis.Walker at thomsonreuters.com 
 > wrote:

> Will/Julian,
>
> I'm curious what it means to turn off NativeNonEmpty but turn on
> NativeCrossjoin.  I usually think of NativeNonEmpty as the "parent"
> property, meaning if it is off, then none of the child properties
> (NativeCrossjoin, NativeTopCount, etc) have any effect.  Is this the
> wrong way to think about it?  If NativeCrossjoin should turn on native
> regardless of the NativeNonEmpty value, then the proposed change makes
> sense to me.  Also, can you send me which test cases are failing?   
> They
> were attached to one of the previous e-mails in this chain.  Thanks.
>
> Kurt
>
> -----Original Message-----
> From: Julian Hyde [mailto:jhyde at pentaho.com]
> Sent: Monday, April 26, 2010 11:32 PM
> To: 'Will Gorman'; Campbell, Matthew (Hlthcr&Science); Walker, Kurtis
> (Hlthcr&Science)
> Cc: 'Benny Chow'; 'Jake Cornelius'; 'Mondrian developer mailing list'
> Subject: RE: 3.2 NonEmptyCrossJoin Issues
>
> Will,
>
> It looks very reasonable. I am running your change through my suite of
> tests
> now.
>
> I think you should check in before the 12 midnight EDT mondrian
> regression
> suite starts. If my suite shows up a problem, I'll let you know and we
> can
> back out.
>
> Matt/Kurtis,
>
> Can you please comment because it was your change 13151 that added
>
>         if (evaluator.nativeEnabled())
>
> into RolapSchemaReader.getNativeSetEvaluator.
>
> Julian
>
>> -----Original Message-----
>> From: Will Gorman [mailto:wgorman at pentaho.com]
>> Sent: Monday, April 26, 2010 8:16 PM
>> To: Julian Hyde
>> Cc: Benny Chow; Jake Cornelius
>> Subject: RE: 3.2 NonEmptyCrossJoin Issues
>>
>> Hi Julian,
>>
>> I've narrowed the change down to
>> RolapSchemaReader.getNativeSetEvaluator(), it is now checking for
>> evaluator.enableNative() before creating a native evaluator.
>> I was able
>> to get the unit test to pass by changing the default value of
>> RolapEvaluator.enableNative in the constructor from:
>>    MondrianProperties.instance().EnableNativeNonEmpty.get()
>> to:
>>    MondrianProperties.instance().EnableNativeNonEmpty.get() ||
>>    MondrianProperties.instance().EnableNativeCrossJoin.get();
>>
>> All the unit tests pass.  Does that seem like a reasonable
>> approach?  If
>> I just remove the check in RolapSchemaReader there are test failures.
>>
>> Thanks,
>>
>> Will
>>
>> On Mon, 2010-04-26 at 22:15 -0400, Will Gorman wrote:
>>> After running a quick binary search through change lists,
>> I've narrowed
>>> down the regression to change #13151 - "MONDRIAN -
>> NativizeSet - making
>>> sure each high cardinality determination is not affected by the
>>> evaluation of previous axes."
>>>
>>> Also, I don't think the slimmed down unit test is really
>> testing for the
>>> issue, the exception that is thrown is in a different area
>> of the code.
>>>
>>> Will
>>>
>>> On Mon, 2010-04-26 at 17:52 -0400, Will Gorman wrote:
>>>> Hi Julian,
>>>>
>>>> Attached are two unit test cases that demonstrate the
>> issue.  I used the
>>>> original MDX and Benny's slimmed down version for the two
>> test cases.
>>>> Hopefully this helps!
>>>>
>>>> Will
>>>>
>>>> On Sat, 2010-04-24 at 04:59 -0500, Benny Chow wrote:
>>>>> Julian,
>>>>>
>>>>> I *think* the regression was caused by your change
>> 13234 on 12/16/2009
>>>>> 03:02:56 AM MONDRIAN: Rationalize & tune how root evaluator is
>>>>> created.
>>>>>
>>>>> Mondrian's code is complicated and I'm not very clear
>> on what's going
>>>>> on in there.  But looking at the changes in RolapEvaluator I'm
>>>>> guessing that when the properties were set to:
>>>>>
>>>>> mondrian.native.crossjoin.enable=true
>>>>> mondrian.native.nonempty.enable=false
>>>>>
>>>>> Native CJ evaluation was still happening but after
>> 13234, native CJ
>>>>> stopped kicking in.
>>>>>
>>>>> Here's a simple query that should be native evaluated
>> with the above
>>>>> property settings:
>>>>>
>>>>> With
>>>>> Set [*NATIVE_CJ_SET] as 'NonEmptyCrossJoin([Education
>>>>> Level].[Education Level].Members,[Store Type].[Store
>> Type].Members)'
>>>>>
>>>>> Select
>>>>> [*NATIVE_CJ_SET]  on columns
>>>>> From [Sales]
>>>>>
>>>>> Is this enough information for you to try to fix this?
>>>>>
>>>>> Benny
>>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Benny Chow
>>>>> Sent: Sat 4/24/2010 12:24 PM
>>>>> To: Will Gorman
>>>>> Cc: Julian Hyde; Jake Cornelius
>>>>> Subject: RE: 3.2 NonEmptyCrossJoin Issues
>>>>>
>>>>> Hi Will,
>>>>>
>>>>> I checked in the code.  There are currently three test
>> failures.  2
>>>>> are related to Marc's change which I sent email earlier
>> about.  1 is
>>>>> related to the nonemptycrossjoin issue.
>>>>>
>>>>> IMO, it's a blocker since it's a big performance issue.
>> Analyzer MDX
>>>>> generation was architected to use the NECJ to return only valid
>>>>> combinations of attributes on the report.
>>>>>
>>>>> I'll do some more investigation over the weekend.
>>>>>
>>>>> Benny
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Will Gorman
>>>>> Sent: Saturday, April 24, 2010 5:30 AM
>>>>> To: Benny Chow
>>>>> Cc: Julian Hyde; Jake Cornelius
>>>>> Subject: Re: 3.2 NonEmptyCrossJoin Issues
>>>>>
>>>>> Hi Benny,
>>>>>
>>>>> Thanks for doing the research.  Could you check in your
>> progress?
>>>>> I've
>>>>> made similar changes on the reporting side, and
>> verified all JPivot
>>>>> unit
>>>>> tests are passing.
>>>>>
>>>>> Also, do you feel that the native regression is a
>> blocker for our RC
>>>>> release, or are you okay with waiting till GA to have
>> this resolved?
>>>>>
>>>>> Thanks!
>>>>>
>>>>> Will
>>>>>
>>>>> On Fri, 2010-04-23 at 12:06 -0500, Benny Chow wrote:
>>>>>> Julian,
>>>>>>
>>>>>> I refactored Analyzer's code to not depend on the All member
>>>>> anymore
>>>>>> and use getParentMember to find parent members.  The
>> majority of
>>>>>> Analyzer's unit tests pass now.  I think next time
>> you fix old
>>>>>> LucidEra Clearview unit tests, you should give me a
>> heads up since
>>>>> all
>>>>>> of those tests exist in Analyzer.
>>>>>>
>>>>>> The NonEmptyCrossJoin issue I am running into is that native
>>>>>> evaluation is not happening anymore for the below query:
>>>>>>
>>>>>>
>>>>>> With
>>>>>> Set [*NATIVE_CJ_SET] as
>> 'NonEmptyCrossJoin([*BASE_MEMBERS_Education
>>>>>> Level],NonEmptyCrossJoin([*BASE_MEMBERS_Common
>> Date-Canonical Dates
>>>>>> Calendar Hierarchy],[*BASE_MEMBERS_Product]))'
>>>>>>
>>>>>> Set [*BASE_MEMBERS_Common Date-Canonical Dates
>> Calendar Hierarchy]
>>>>> as
>>>>>> '[Common Date-Canonical Dates Calendar Hierarchy].[Calendar
>>>>>> Month].Members'
>>>>>>
>>>>>> Set [*BASE_MEMBERS_Education Level] as '[Education
>>>>> Level].[Education
>>>>>> Level].Members'
>>>>>> Set [*BASE_MEMBERS_Product] as '[Product].[Product
>> Family].Members'
>>>>>> Set [*BASE_MEMBERS_Measures] as
>>>>> '{[Measures].[*FORMATTED_MEASURE_0]}'
>>>>>> Member [Measures].[*FORMATTED_MEASURE_0] as
>> '[Measures].[Sales
>>>>>> Count]', FORMAT_STRING = '#,###', SOLVE_ORDER=400
>>>>>> Select
>>>>>> [*BASE_MEMBERS_Measures] on columns,
>>>>>> Non Empty [*NATIVE_CJ_SET] on rows
>>>>>> From [Sales]
>>>>>>
>>>>>>
>>>>>> The [*NATIVE_CJ_SET] should have been natively
>> evaluated with all
>>>>>> three dimensions pushed down to the DB.
>>>>>>
>>>>>> The current mondrian.properties is the same as I sent you
>>>>> previously.
>>>>>> I think the two key properties are:
>>>>>>
>>>>>> # if enabled some NON EMPTY CrossJoin will be computed in SQL
>>>>>>
>>>>>> mondrian.native.crossjoin.enable=true
>>>>>>
>>>>>> # some NON EMPTY set operations like member.children,
>> level.members
>>>>>> and member descendants will be computed in SQL
>>>>>>
>>>>>> mondrian.native.nonempty.enable=false
>>>>>>
>>>>>>
>>>>>> These properties have not changed so the behavior
>> should not have
>>>>>> changed either.
>>>>>>
>>>>>> Analyzer has a bunch of unit tests that test for
>> native evaulation
>>>>> by
>>>>>> turning on
>>>>>>
>> MondrianProperties.instance().AlertNativeEvaluationUnsupported.
>>>>> (This
>>>>>> was something JVS added). But because of this
>> regression, Mondrian
>>>>> is
>>>>>> not reporting when native evaluation is not being used (but
>>>>> expected).
>>>>>>
>>>>>> I'm still investigating this but if you know that
>> code has been
>>>>>> changed in this area you can give me some pointers...
>>>>>>
>>>>>> Thanks
>>>>>> Benny
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>>
>
>



More information about the Mondrian mailing list