[Mondrian] proposal for Order function with key specification list

Rushan Chen rchen at lucidera.com
Thu Sep 25 13:13:15 EDT 2008


> I believe that Marc is considering making some changes to the Order
> function: to use a multi-threaded sort algorithm; and optimizing the sort if
> the results are going to be used for TopCount.
>
> Rushan and Marc, I don't know when you are thinking of working on this, but
> you should coordinate to make sure that you are not treading on each other's
> feet.
>
>   
Thanks for the heads-up.

We are working on the Order function extension at this moment and it 
might take the next couple of weeks to have it completed. Currently we 
are developing it in our private branch to test the feature out.

When we integrate, we will make sure enough tests are checked in to make 
sure the feature can be verified by anyone working in the same area.
>> For function with native evaluator, conformance in native mode could 
>> actually make native evaluation not very useful. For example, NECJ of 
>> two enumerated sets of arbitrary order. For these cases, if the 
>> application does not provide additional information regarding the 
>> expected sort order, we could in fact turn off native evaluation.
>>     
>
> Rushan, I think you're saying that mondrian would need to disable native
> mode in cases where the query doesn't sort the result (or use Unorder to say
> that it doesn't care), and if so, I agree. It's a shame that native mode
> would get used less, but if people want mondrian to use native mode, it's
> easy to achieve: just specify the full ordering you want. Ironically, this
> is already a best practice for people writing SQL.
>
> Julian
>
>   
That is what I think should happen. Regarding native evaluation having 
to be turned off for some queries, I hesitate to suggest this, but we 
could provide a parameter to be used by those applications which can not 
be easily changed to include result ordering but reply on native 
evaluation that does not conform to the order preservedness of the 
function/expression being natively evaluated.
>> -----Original Message-----
>> From: Rushan Chen [mailto:rchen at lucidera.com] 
>> Sent: Friday, September 19, 2008 7:50 PM
>> To: jhyde at pentaho.com; Mondrian developer mailing list
>> Cc: 'John V. Sichi'
>> Subject: Re: [Mondrian] proposal for Order function with key 
>> specification list
>>
>> (Back from vacation and am catching up with the emails...)
>>
>> Thanks for the many good discussions on this topic in the 
>> forum. A few 
>> response to the suggestions so far:
>>
>> (1) Regarding using hierarchical sort keys in an order different from 
>> the hierarchy order, as in the example
>>
>> Order([Time].members, Ancestor([Time].CurrentMember, [Time].[Year]), 
>> ASC, Ancestor([Time].CurrentMember, [Time].[Year]), ASC)
>>
>> Obviously the intention of the user is to sort first on "quarter" and 
>> then on "year"; so from Julian's suggestions, this should be 
>> used instead:
>>
>> Order([Time].members, Ancestor([Time].CurrentMember, [Time].[Year]), 
>> BASC, Ancestor([Time].CurrentMember, [Time].[Year]), BASC)
>>
>> Also, on the topic of whether it is useful to combine BASC/BDESC and 
>> ASC/DESC, it might be useful for keys from different hierarchies, but 
>> not much otherwise.
>>
>> Order(CrossJoin([Time].members, [Product].members), 
>> Ancestor([Time].CurrentMember, [Time].[Year]), ASC, 
>> [Product].currentMember, BASC)
>>
>> (2) Order preservedness of set functions.
>>
>> For the new Order function, we will declare the order 
>> preservedness in 
>> the document.
>>
>> For function with native evaluator, conformance in native mode could 
>> actually make native evaluation not very useful. For example, NECJ of 
>> two enumerated sets of arbitrary order. For these cases, if the 
>> application does not provide additional information regarding the 
>> expected sort order, we could in fact turn off native evaluation.
>>
>> BTW, Julian, this looks wrong:
>>
>> Order(<set>, <key1>, <dir1>, <key2>, <dir2>)
>>
>> should always be equivalent to
>>
>> Order(
>>   Order(
>>     <set>,
>>     <key2>, <dir2>),
>>   <key1>, <dir1>)
>>
>> Shouldn't the equivalent expression be this?
>>
>> Order(
>>   Order(
>>     <set>,
>>     <key1>, <dir1>),
>>   <key2>, <dir2>)
>>
>>
>> The eigenpedia doc now contains the latest information:
>> http://pub.eigenbase.org/wiki/MondrianOrderFunctionExtension
>>
>> Rushan
>>
>> Julian Hyde wrote:
>>     
>>>   
>>>       
>>>> Currently native NonEmptyCrossJoin and Filter (and maybe 
>>>> others) are out 
>>>> of conformance, and native is enabled by default, but 
>>>> conformance can be 
>>>> achieved by disabling native.
>>>>     
>>>>         
>>> I can allow that in the short term. However I would like 
>>>       
>> mondrian to be in
>>     
>>> conformance even in native mode. (I see native mode as an 
>>>       
>> option that
>>     
>>> affects physical behavior, and hence performance, not 
>>>       
>> logical behavior.) To
>>     
>>> use the efficient NonEmptyCrossJoin and Filter, either 
>>>       
>> applications will
>>     
>>> have to put in Unorder, or the compiler will have to get smarter to
>>> recognize when an enclosing Order means that the underlying 
>>>       
>> function is free
>>     
>>> to return the results in whatever order it wants.
>>>
>>> I've just checked in an Unorder function. It does nothing 
>>>       
>> right now, but
>>     
>>> people can use it for optimization.
>>>
>>> Julian
>>>
>>>
>>> _______________________________________________
>>> Mondrian mailing list
>>> Mondrian at pentaho.org
>>> http://lists.pentaho.org/mailman/listinfo/mondrian
>>>   
>>>       
>> -- 
>> Rushan Chen
>>
>> www.lucidera.com<http://www.lucidera.com/>
>>
>> Gartner: LucidEra is a business intelligence "Cool Vendor
>> <http://www.lucidera.com/company/press_releases/cool_Vendor_le
>> ading_analyst_firm_9apr.php>
>>
>> Beagle Research: LucidEra is a CRM WizKid
>> <http://www.lucidera.com/company/press_releases/lucidera_crm_w
>> izkid_awards_26mar.php>
>>
>> JMP Securities: LucidEra is in the Hot 100
>> <http://www.lucidera.com/company/press_releases/jmp_hot_100_be
>>     
> st_apr2.php>
>   
>> --------------------------------------------------------------
>> -----------------
>> This message and any files or text attached to it are 
>> intended only for
>> the recipients named above, and contain information that may be
>> confidential or privileged. If you are not an intended 
>> recipient, or you
>> have reason to believe that you received this message in 
>> error, you must
>> not read, copy, use or disclose this communication. Please also notify
>> the sender by replying to this message, and then delete all 
>> copies of it
>> from your system. Thank you.
>>
>>
>>
>>
>>
>>     
>
>   






More information about the Mondrian mailing list