[Mondrian] Native expressions - allowing aggregated members on nativetopcount

Pedro Alves pmgalves at gmail.com
Fri Aug 31 10:43:24 EDT 2012



Ok, I got it working. Pedro Vale says Julian's going to get himself on a 
plane to slap me as soon as he sees it :p


The code does need proper review from someone that knows what he's 
doing. But jokes apart, this is the kind of path that will lead to a 
huge improvement in the way high cardinality is dealt within mondrian. 
Native evaluation can put a query from 5 minutes into 5 seconds. And 
this is definitely something we all need


Here's the current version of the code:

https://github.com/webdetails/mondrian/commit/37ba6f65c0db1e07a7d85ff0df60e37eb3a118dc


(I'm running the unit tests to see if everything works, so I may need to 
change a few other things)


The basic approach is allow some calculated members to pass to the 
native evaluation. Currently I'm allowing Aggregates, but probably this 
could apply to other parts.


There are a few things that I'm not too sure about.

1. The function getSetFromCalculatedMember;  I'm looking for the first 
argument of the aggregate funcion, and extract the named set. I'm sure 
there's an easier way just do evaluate that part and look at the result 
- if it's a Member[], cool. Would allow us not to have to hardwire the 
list of supported functions


2. In addContextConstraint, there's something that doesn't make me very 
comfortable, the way I had to implement 
getUniqueOrdinalMembers(members). If I don't do that, CellRequest comes 
out as unsatisfiable, even if later on the constraint can be made out of 
those multiple members. But in my defense, this is exactly what happens 
when we're taking a set out of the slicer and building the 
evaluator.setContext - we only get one random member for each distinct 
ordinal



Let me know - I'll try to fix the errors in the tests, but any pointers 
would be greatly appreciated



-pedro





On 08/31/2012 05:15 AM, Brandon Jackson wrote:
> Very cool.  It's great to see solutions evolving on all of these
> important fronts.
>
> Good luck and sincere thanks.
>
> The hammer eh?
>
> Sent from my iPhone
>
> On Aug 30, 2012, at 8:28 PM, Pedro Alves <pmgalves at gmail.com
> <mailto:pmgalves at gmail.com>> wrote:
>
>> Tomorrow I hope to have a working code with native top counts working
>> with simple aggregations!!
>>
>>
>> The code will need review and cleanup but I'm absolutely thrilled
>> because for the first time I'm actually doing something that goes
>> beyhond the one line fix in mondrian and will have huge impact on high
>> carninality dimensions \o/ \o/
>>
>>
>> - pedro "the hammer" alves
>>
>>
>>
>> On Friday, August 31, 2012, Julian Hyde wrote:
>>
>>     We'd always like to push more stuff down for native SQL
>>     evaluation. But pushing a general aggregation expression down is
>>     hard (the expressions don't often turn into clean SQL). If there
>>     are only a few members in the slicer, the code in executeStrip
>>     isn't too bad.
>>
>>     Julian
>>
>>
>>     On Aug 29, 2012, at 11:37 AM, Pedro Alves <pmgalves at gmail.com
>>     <javascript:;>> wrote:
>>
>>     >
>>     >
>>     > My goal with my previous bug is to allow native top counts to be
>>     used
>>     > with sets.
>>     >
>>     >
>>     > And I'm a bit lost here...
>>     >
>>     >
>>     > AA)
>>     >
>>     > this seems natural to me:
>>     >
>>     > with set a as '([Time].[1997].[Q1] : [Time].[1997].[Q2])'
>>     > member Time.x as Aggregate(a,[Measures].[Store Sales])
>>     > member Measures.x1 as ([Time].[1997].[Q1],[Measures].[Store Sales])
>>     > member Measures.x2 as ([Time].[1997].[Q2],[Measures].[Store Sales])
>>     > set products as TopCount(Product.[Product
>>     > Name].Members,2,Measures.[Store Sales])
>>     > SELECT
>>     > NON EMPTY products ON 1,
>>     > NON EMPTY {[Measures].[Store Sales], Measures.x1, Measures.x2} ON 0
>>     > FROM [Sales]
>>     > -- where ([Time].[1997].[Q1] : [Time].[1997].[Q2])
>>     > where Time.x
>>     >
>>     >
>>     > but native evaluations don't allow for calculated members. Would
>>     it be a
>>     > good idea to try to support aggregates to be used in native
>>     evaluations?
>>     >
>>     >
>>     >
>>     > BB)
>>     >
>>     > with set a as '([Time].[1997].[Q1] : [Time].[1997].[Q2])'
>>     > member Time.x as Aggregate(a,[Measures].[Store Sales])
>>     > member Measures.x1 as ([Time].[1997].[Q1],[Measures].[Store Sales])
>>     > member Measures.x2 as ([Time].[1997].[Q2],[Measures].[Store Sales])
>>     > set products as TopCount(Product.[Product
>>     > Name].Members,2,Measures.[Store Sales])
>>     > SELECT
>>     > NON EMPTY products ON 1,
>>     > NON EMPTY {[Measures].[Store Sales], Measures.x1, Measures.x2} ON 0
>>     > FROM [Sales]
>>     > where ([Time].[1997].[Q1] : [Time].[1997].[Q2])
>>     > -- where Time.x
>>     >
>>     > This works... well, apart from the bug I mentioned before. But I
>>     don't
>>     > like it a lot; RolapResults:executeStripe does this:
>>     >
>>     >             RolapAxis axis = (RolapAxis) slicerAxis;
>>     >             TupleList tupleList = axis.getTupleList();
>>     >             for (List<Member> members : tupleList) {
>>     >
>>     > And will do the entire evaluation for every member of that
>>     tupleList.
>>     > That's a lot of processing power.
>>     >
>>     >
>>     > CC)
>>     >
>>     > Is there any other way of doing this queries that I'm not
>>     seeing? Seems
>>     > to me that I'm over complicating things when all I want is to
>>     start my
>>     > query with a calculated set that I want to be natively evaluated...
>>     >
>>     >
>>     >
>>     >
>>     > -pedro
>>     >
>>     >
>>     >
>>     > _______________________________________________
>>     > Mondrian mailing list
>>     > Mondrian at pentaho.org <javascript:;>
>>     > http://lists.pentaho.org/mailman/listinfo/mondrian
>>
>>     _______________________________________________
>>     Mondrian mailing list
>>     Mondrian at pentaho.org <javascript:;>
>>     http://lists.pentaho.org/mailman/listinfo/mondrian
>>
>> _______________________________________________
>> Mondrian mailing list
>> Mondrian at pentaho.org <mailto:Mondrian at pentaho.org>
>> http://lists.pentaho.org/mailman/listinfo/mondrian
>
>
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>


More information about the Mondrian mailing list