[Mondrian] Re: Test failures with high-cardinality dimensions

[Luis F. Canals]

Hi all,

about this question, problem arises when using functions with high  
cardinality dimensions not implemented (or not implementable) for this  
kind of dimensions.
Since not every function or query will work for high cardinality  
dimensions, I think it's not a good idea to use  
"mondrian.test.highCardDimensions=...". The solution would be to use  
mondrian/rolap/HighDimensionsTest (or simiar) and include any tests  
related with high dimensions into it instead of this kind of "blind"  
tests for all functions.

So the first thing to do is: move from testing every functions against  
high card. dimensions to detailed tests function by function for  
suitable ones.

The key question here is the limitations for the meaning of "high  
cardility" feature: high cardinality dimensions are able to be  
managed. But the use of functions with these kind of dimensions can't  
be equivalent to usual dimensions. Mainly, all functions that require  
the whole set of elements to produce a result will not work for high  
cardinality dimensions (obviously).

At this time, we have no human sructure to refactor everything and  
implement all possible functions to work with high dimension property  
(Mondrian is a very big project).
It's a continuous task, open for everyone, ans sould be done by  
everyone.

The first (and most difficult) step has been done: Mondrian can read  
high cardinality dimensions; the second step (implement all suitable  
functions to work for high cardimality dimensions) is drafted, with  
"head", "count", "filter" and "non empty" functions already running.

I see here an opportunity to "transfer knowledge": it would be better  
that someone (different than us) made propper changes to implement  
"Ancestor" function (if has sense) for high cardilanity dimensions.  
For example, for "Ancestor" function, the approach would be:
    The first question: Would Ancestor work for high cardinality  
dimensions?
    Second question: How would it work for h.c. dimensions?
    Then, implement it.


Best regards!


- Luis F. Canals
   CISM
   luis.canals at stratebi.com



On 06/07/2008, at 19:48, Julian Hyde wrote:

> Luis, Jorge, Javier,
>
> There are still some errors with high-cardinality dimensions. See  
> below; and also search for 'execHighCardTest' in http://www.hydromatic.net/buildlogs/mondrian-marmalade.hydromatic.net-20080705-change11263.log.bz2 
> .
>
> Do you know about these? Any idea what the cause is? When can they  
> be fixed?
>
> Julian
>
> Running test with JDK=jdk1.6 retroweave= database=oracle  
> props={ mondrian.test.highCardDimensions=Store}
>
> [java] 1)  
> testAncestorNumeric 
> (mondrian.olap.fun.FunctionTest)mondrian.olap.MondrianException:  
> Mondrian Error:Failed to parse query 'select {Ancestor([Store].[USA]. 
> [Washington], 1)} on columns from [Sales Ragged]'
>
> [java] 2)  
> testAncestorWithHiddenParent 
> (mondrian.olap.fun.FunctionTest)mondrian.olap.MondrianException:  
> Mondrian Error:Failed to parse query 'select {Ancestor([Store].[All  
> Stores].[Israel].[Haifa], [Store].[Store Country])} on columns from  
> [Sales Ragged]'
>
> [java] 3)  
> testOrdinal 
> (mondrian.olap.fun.FunctionTest)mondrian.olap.MondrianException:  
> Mondrian Error:Failed to parse query 'with member [Measures].[Foo]  
> as '[Store].[All Stores].[USA].[Washington].ordinal' select  
> {[Measures].[Foo]} on columns from [Sales Ragged]'
>
> [java] 4)  
> testLead 
> (mondrian.test.RaggedHierarchyTest)mondrian.olap.MondrianException:  
> Mondrian Error:Failed to parse query 'select {[Store].[All Stores]. 
> [Mexico].[DF].Lead(1)} on columns from [Sales Ragged]'
>
> [java] 5)  
> testParentOfHaifa 
> (mondrian.test.RaggedHierarchyTest)mondrian.olap.MondrianException:  
> Mondrian Error:Failed to parse query 'select {[Store].[Israel]. 
> [Haifa].Parent} on columns from [Sales Ragged]'
>
> [java] 6)  
> testPrevMemberOfHaifa 
> (mondrian.test.RaggedHierarchyTest)mondrian.olap.MondrianException:  
> Mondrian Error:Failed to parse query 'select {[Store].[Israel]. 
> [Haifa].PrevMember} on columns from [Sales Ragged]'
>
> [java] 7)  
> testNextMemberOfTelAviv 
> (mondrian.test.RaggedHierarchyTest)mondrian.olap.MondrianException:  
> Mondrian Error:Failed to parse query 'select {[Store].[Israel].[Tel  
> Aviv].NextMember} on columns from [Sales Ragged]'
>
> [java] 8)  
> testNextMemberOfBC 
> (mondrian.test.RaggedHierarchyTest)mondrian.olap.MondrianException:  
> Mondrian Error:Failed to parse query 'select {[Store].[All Stores]. 
> [Canada].[BC].NextMember} on columns from [Sales Ragged]'
>
> [java] 9)  
> testAncestorOfHaifa 
> (mondrian.test.RaggedHierarchyTest)mondrian.olap.MondrianException:  
> Mondrian Error:Failed to parse query 'select {Ancestor([Store]. 
> [Israel].[Haifa], [Store].[Store State])} on columns from [Sales  
> Ragged]'
>
> [java] 10)  
> testDataSourceChangeListenerPlugin 
> (mondrian 
> .rolap.DataSourceChangeListenerTest)java.lang.ClassCastException:  
> mondrian.rolap.NoCacheMemberReader cannot be cast to  
> mondrian.rolap.SmartMemberReader
>
> [java] 11)  
> testLookupMemberCache 
> (mondrian.rolap.NonEmptyTest)java.lang.ClassCastException:  
> mondrian.rolap.NoCacheMemberReader cannot be cast to  
> mondrian.rolap.SmartMemberReader
>
> [java] 12)  
> testLookupMember2 
> (mondrian.rolap.NonEmptyTest)mondrian.olap.MondrianException:  
> Mondrian Error:Failed to parse query 'select {[Store].[USA]. 
> [Washington]} on columns from [Sales Ragged]'
>
> [java] 1)  
> testCrossJoinAsteriskQuery 
> (mondrian.olap.fun.FunctionTest)junit.framework.ComparisonFailure:  
> Expected:
>
> [java] 2)  
> testCalculatedMemberInCube 
> (mondrian 
> .test.TestCalculatedMembers)junit.framework.ComparisonFailure:  
> Expected:
>
> [java] 3)  
> testHierarchize 
> (mondrian 
> .test.RaggedHierarchyTest)junit.framework.ComparisonFailure: Expected:
>
> [java] 4)  
> testDescendantsOfRootAtCity 
> (mondrian 
> .test.RaggedHierarchyTest)junit.framework.ComparisonFailure: Expected:
>
> [java] Tests run: 1856, Failures: 4, Errors: 12
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20080714/d2734a4f/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://lists.pentaho.org/pipermail/mondrian/attachments/20080714/d2734a4f/attachment.bin