[Mondrian] Re: Test failures with high-cardinality dimensions

Luis F. Canals luis.canals at stratebi.com
Mon Jul 14 05:47:16 EDT 2008


(Signed messages arenot easy readable.... sorry....)

Hi all,

about this question, problem arises when using functions with high  
cardinality dimensions not implemented (or not implementable) for this  
kind of dimensions.
Since not every function or query will work for high cardinality  
dimensions, I think it's not a good idea to use  
"mondrian.test.highCardDimensions=...". The solution would be to use  
mondrian/rolap/HighDimensionsTest (or simiar) and include any tests  
related with high dimensions into it instead of this kind of "blind"  
tests for all functions.

So the first thing to do is: move from testing every functions against  
high card. dimensions to detailed tests function by function for  
suitable ones.

The key question here is the limitations for the meaning of "high  
cardility" feature: high cardinality dimensions are able to be  
managed. But the use of functions with these kind of dimensions can't  
be equivalent to usual dimensions. Mainly, all functions that require  
the whole set of elements to produce a result will not work for high  
cardinality dimensions (obviously).

At this time, we have no human sructure to refactor everything and  
implement all possible functions to work with high dimension property  
(Mondrian is a very big project).
It's a continuous task, open for everyone, ans sould be done by  
everyone.

The first (and most difficult) step has been done: Mondrian can read  
high cardinality dimensions; the second step (implement all suitable  
functions to work for high cardimality dimensions) is drafted, with  
"head", "count", "filter" and "non empty" functions already running.

I see here an opportunity to "transfer knowledge": it would be better  
that someone (different than us) made propper changes to implement  
"Ancestor" function (if has sense) for high cardilanity dimensions.  
For example, for "Ancestor" function, the approach would be:
    The first question: Would Ancestor work for high cardinality  
dimensions?
    Second question: How would it work for h.c. dimensions?
    Then, implement it.


Best regards!


On 14/07/2008, at 8:47, Luis F. Canals wrote:

> Hi all,
>
> about this question, problem arises when using functions with high  
> cardinality dimensions not implemented (or not implementable) for  
> this kind of dimensions.
> Since not every function or query will work for high cardinality  
> dimensions, I think it's not a good idea to use  
> "mondrian.test.highCardDimensions=...". The solution would be to use  
> mondrian/rolap/HighDimensionsTest (or simiar) and include any tests  
> related with high dimensions into it instead of this kind of "blind"  
> tests for all functions.
>
> So the first thing to do is: move from testing every functions  
> against high card. dimensions to detailed tests function by function  
> for suitable ones.
>
> The key question here is the limitations for the meaning of "high  
> cardility" feature: high cardinality dimensions are able to be  
> managed. But the use of functions with these kind of dimensions  
> can't be equivalent to usual dimensions. Mainly, all functions that  
> require the whole set of elements to produce a result will not work  
> for high cardinality dimensions (obviously).
>
> At this time, we have no human sructure to refactor everything and  
> implement all possible functions to work with high dimension  
> property (Mondrian is a very big project).
> It's a continuous task, open for everyone, ans sould be done by  
> everyone.
>
> The first (and most difficult) step has been done: Mondrian can read  
> high cardinality dimensions; the second step (implement all suitable  
> functions to work for high cardimality dimensions) is drafted, with  
> "head", "count", "filter" and "non empty" functions already running.
>
> I see here an opportunity to "transfer knowledge": it would be  
> better that someone (different than us) made propper changes to  
> implement "Ancestor" function (if has sense) for high cardilanity  
> dimensions. For example, for "Ancestor" function, the approach would  
> be:
>    The first question: Would Ancestor work for high cardinality  
> dimensions?
>    Second question: How would it work for h.c. dimensions?
>    Then, implement it.
>
>
> Best regards!
>
>
> - Luis F. Canals
>   CISM
>   luis.canals at stratebi.com
>
>
>
> On 06/07/2008, at 19:48, Julian Hyde wrote:
>
>> Luis, Jorge, Javier,
>>
>> There are still some errors with high-cardinality dimensions. See  
>> below; and also search for 'execHighCardTest' in http://www.hydromatic.net/buildlogs/mondrian-marmalade.hydromatic.net-20080705-change11263.log.bz2 
>> .
>>
>> Do you know about these? Any idea what the cause is? When can they  
>> be fixed?
>>
>> Julian
>>
>> Running test with JDK=jdk1.6 retroweave= database=oracle  
>> props={ mondrian.test.highCardDimensions=Store}
>>
>> [java] 1)  
>> testAncestorNumeric 
>> (mondrian.olap.fun.FunctionTest)mondrian.olap.MondrianException:  
>> Mondrian Error:Failed to parse query 'select {Ancestor([Store]. 
>> [USA].[Washington], 1)} on columns from [Sales Ragged]'
>>
>> [java] 2)  
>> testAncestorWithHiddenParent 
>> (mondrian.olap.fun.FunctionTest)mondrian.olap.MondrianException:  
>> Mondrian Error:Failed to parse query 'select {Ancestor([Store].[All  
>> Stores].[Israel].[Haifa], [Store].[Store Country])} on columns from  
>> [Sales Ragged]'
>>
>> [java] 3)  
>> testOrdinal 
>> (mondrian.olap.fun.FunctionTest)mondrian.olap.MondrianException:  
>> Mondrian Error:Failed to parse query 'with member [Measures].[Foo]  
>> as '[Store].[All Stores].[USA].[Washington].ordinal' select  
>> {[Measures].[Foo]} on columns from [Sales Ragged]'
>>
>> [java] 4)  
>> testLead 
>> (mondrian.test.RaggedHierarchyTest)mondrian.olap.MondrianException:  
>> Mondrian Error:Failed to parse query 'select {[Store].[All Stores]. 
>> [Mexico].[DF].Lead(1)} on columns from [Sales Ragged]'
>>
>> [java] 5)  
>> testParentOfHaifa 
>> (mondrian.test.RaggedHierarchyTest)mondrian.olap.MondrianException:  
>> Mondrian Error:Failed to parse query 'select {[Store].[Israel]. 
>> [Haifa].Parent} on columns from [Sales Ragged]'
>>
>> [java] 6)  
>> testPrevMemberOfHaifa 
>> (mondrian.test.RaggedHierarchyTest)mondrian.olap.MondrianException:  
>> Mondrian Error:Failed to parse query 'select {[Store].[Israel]. 
>> [Haifa].PrevMember} on columns from [Sales Ragged]'
>>
>> [java] 7)  
>> testNextMemberOfTelAviv 
>> (mondrian.test.RaggedHierarchyTest)mondrian.olap.MondrianException:  
>> Mondrian Error:Failed to parse query 'select {[Store].[Israel].[Tel  
>> Aviv].NextMember} on columns from [Sales Ragged]'
>>
>> [java] 8)  
>> testNextMemberOfBC 
>> (mondrian.test.RaggedHierarchyTest)mondrian.olap.MondrianException:  
>> Mondrian Error:Failed to parse query 'select {[Store].[All Stores]. 
>> [Canada].[BC].NextMember} on columns from [Sales Ragged]'
>>
>> [java] 9)  
>> testAncestorOfHaifa 
>> (mondrian.test.RaggedHierarchyTest)mondrian.olap.MondrianException:  
>> Mondrian Error:Failed to parse query 'select {Ancestor([Store]. 
>> [Israel].[Haifa], [Store].[Store State])} on columns from [Sales  
>> Ragged]'
>>
>> [java] 10)  
>> testDataSourceChangeListenerPlugin 
>> (mondrian 
>> .rolap.DataSourceChangeListenerTest)java.lang.ClassCastException:  
>> mondrian.rolap.NoCacheMemberReader cannot be cast to  
>> mondrian.rolap.SmartMemberReader
>>
>> [java] 11)  
>> testLookupMemberCache 
>> (mondrian.rolap.NonEmptyTest)java.lang.ClassCastException:  
>> mondrian.rolap.NoCacheMemberReader cannot be cast to  
>> mondrian.rolap.SmartMemberReader
>>
>> [java] 12)  
>> testLookupMember2 
>> (mondrian.rolap.NonEmptyTest)mondrian.olap.MondrianException:  
>> Mondrian Error:Failed to parse query 'select {[Store].[USA]. 
>> [Washington]} on columns from [Sales Ragged]'
>>
>> [java] 1)  
>> testCrossJoinAsteriskQuery 
>> (mondrian.olap.fun.FunctionTest)junit.framework.ComparisonFailure:  
>> Expected:
>>
>> [java] 2)  
>> testCalculatedMemberInCube 
>> (mondrian 
>> .test.TestCalculatedMembers)junit.framework.ComparisonFailure:  
>> Expected:
>>
>> [java] 3)  
>> testHierarchize 
>> (mondrian 
>> .test.RaggedHierarchyTest)junit.framework.ComparisonFailure:  
>> Expected:
>>
>> [java] 4)  
>> testDescendantsOfRootAtCity 
>> (mondrian 
>> .test.RaggedHierarchyTest)junit.framework.ComparisonFailure:  
>> Expected:
>>
>> [java] Tests run: 1856, Failures: 4, Errors: 12
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20080714/bb7cc7c3/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://lists.pentaho.org/pipermail/mondrian/attachments/20080714/bb7cc7c3/attachment.bin 


More information about the Mondrian mailing list