[Mondrian] proposal for Order function with key specification list

Rushan Chen rchen at lucidera.com
Wed Oct 15 04:27:19 EDT 2008


The new Order function is almost ready except that we found out about a 
potentially ambiguous interpretation of the existing syntax.  It seems 
we need to modify the syntax to remove the double-meaning.

This section describes the problem and two non-ambiguous syntax choices:

http://pub.eigenbase.org/wiki/MondrianOrderFunctionExtension#Syntax_Discussion

Any one with use cases for or against either of them?

Thanks,

Rushan


Rushan Chen wrote:
>
>> I believe that Marc is considering making some changes to the Order
>> function: to use a multi-threaded sort algorithm; and optimizing the 
>> sort if
>> the results are going to be used for TopCount.
>>
>> Rushan and Marc, I don't know when you are thinking of working on 
>> this, but
>> you should coordinate to make sure that you are not treading on each 
>> other's
>> feet.
>>
>>   
> Thanks for the heads-up.
>
> We are working on the Order function extension at this moment and it 
> might take the next couple of weeks to have it completed. Currently we 
> are developing it in our private branch to test the feature out.
>
> When we integrate, we will make sure enough tests are checked in to 
> make sure the feature can be verified by anyone working in the same area.
>>> For function with native evaluator, conformance in native mode could 
>>> actually make native evaluation not very useful. For example, NECJ 
>>> of two enumerated sets of arbitrary order. For these cases, if the 
>>> application does not provide additional information regarding the 
>>> expected sort order, we could in fact turn off native evaluation.
>>>     
>>
>> Rushan, I think you're saying that mondrian would need to disable native
>> mode in cases where the query doesn't sort the result (or use Unorder 
>> to say
>> that it doesn't care), and if so, I agree. It's a shame that native mode
>> would get used less, but if people want mondrian to use native mode, 
>> it's
>> easy to achieve: just specify the full ordering you want. Ironically, 
>> this
>> is already a best practice for people writing SQL.
>>
>> Julian
>>
>>   
> That is what I think should happen. Regarding native evaluation having 
> to be turned off for some queries, I hesitate to suggest this, but we 
> could provide a parameter to be used by those applications which can 
> not be easily changed to include result ordering but reply on native 
> evaluation that does not conform to the order preservedness of the 
> function/expression being natively evaluated.
>>> -----Original Message-----
>>> From: Rushan Chen [mailto:rchen at lucidera.com] Sent: Friday, 
>>> September 19, 2008 7:50 PM
>>> To: jhyde at pentaho.com; Mondrian developer mailing list
>>> Cc: 'John V. Sichi'
>>> Subject: Re: [Mondrian] proposal for Order function with key 
>>> specification list
>>>
>>> (Back from vacation and am catching up with the emails...)
>>>
>>> Thanks for the many good discussions on this topic in the forum. A 
>>> few response to the suggestions so far:
>>>
>>> (1) Regarding using hierarchical sort keys in an order different 
>>> from the hierarchy order, as in the example
>>>
>>> Order([Time].members, Ancestor([Time].CurrentMember, [Time].[Year]), 
>>> ASC, Ancestor([Time].CurrentMember, [Time].[Year]), ASC)
>>>
>>> Obviously the intention of the user is to sort first on "quarter" 
>>> and then on "year"; so from Julian's suggestions, this should be 
>>> used instead:
>>>
>>> Order([Time].members, Ancestor([Time].CurrentMember, [Time].[Year]), 
>>> BASC, Ancestor([Time].CurrentMember, [Time].[Year]), BASC)
>>>
>>> Also, on the topic of whether it is useful to combine BASC/BDESC and 
>>> ASC/DESC, it might be useful for keys from different hierarchies, 
>>> but not much otherwise.
>>>
>>> Order(CrossJoin([Time].members, [Product].members), 
>>> Ancestor([Time].CurrentMember, [Time].[Year]), ASC, 
>>> [Product].currentMember, BASC)
>>>
>>> (2) Order preservedness of set functions.
>>>
>>> For the new Order function, we will declare the order preservedness 
>>> in the document.
>>>
>>> For function with native evaluator, conformance in native mode could 
>>> actually make native evaluation not very useful. For example, NECJ 
>>> of two enumerated sets of arbitrary order. For these cases, if the 
>>> application does not provide additional information regarding the 
>>> expected sort order, we could in fact turn off native evaluation.
>>>
>>> BTW, Julian, this looks wrong:
>>>
>>> Order(<set>, <key1>, <dir1>, <key2>, <dir2>)
>>>
>>> should always be equivalent to
>>>
>>> Order(
>>>   Order(
>>>     <set>,
>>>     <key2>, <dir2>),
>>>   <key1>, <dir1>)
>>>
>>> Shouldn't the equivalent expression be this?
>>>
>>> Order(
>>>   Order(
>>>     <set>,
>>>     <key1>, <dir1>),
>>>   <key2>, <dir2>)
>>>
>>>
>>> The eigenpedia doc now contains the latest information:
>>> http://pub.eigenbase.org/wiki/MondrianOrderFunctionExtension
>>>
>>> Rushan
>>>
>>> Julian Hyde wrote:
>>>    
>>>>        
>>>>> Currently native NonEmptyCrossJoin and Filter (and maybe others) 
>>>>> are out of conformance, and native is enabled by default, but 
>>>>> conformance can be achieved by disabling native.
>>>>>             
>>>> I can allow that in the short term. However I would like       
>>> mondrian to be in
>>>    
>>>> conformance even in native mode. (I see native mode as an       
>>> option that
>>>    
>>>> affects physical behavior, and hence performance, not       
>>> logical behavior.) To
>>>    
>>>> use the efficient NonEmptyCrossJoin and Filter, either       
>>> applications will
>>>    
>>>> have to put in Unorder, or the compiler will have to get smarter to
>>>> recognize when an enclosing Order means that the underlying       
>>> function is free
>>>    
>>>> to return the results in whatever order it wants.
>>>>
>>>> I've just checked in an Unorder function. It does nothing       
>>> right now, but
>>>    
>>>> people can use it for optimization.
>>>>
>>>> Julian
>>>>
>>>>
>>>> _______________________________________________
>>>> Mondrian mailing list
>>>> Mondrian at pentaho.org
>>>> http://lists.pentaho.org/mailman/listinfo/mondrian
>>>>         
>>> -- 
>>> Rushan Chen
>>>
>>> www.lucidera.com<http://www.lucidera.com/>
>>>
>>> Gartner: LucidEra is a business intelligence "Cool Vendor
>>> <http://www.lucidera.com/company/press_releases/cool_Vendor_le
>>> ading_analyst_firm_9apr.php>
>>>
>>> Beagle Research: LucidEra is a CRM WizKid
>>> <http://www.lucidera.com/company/press_releases/lucidera_crm_w
>>> izkid_awards_26mar.php>
>>>
>>> JMP Securities: LucidEra is in the Hot 100
>>> <http://www.lucidera.com/company/press_releases/jmp_hot_100_be
>>>     
>> st_apr2.php>
>>  
>>> --------------------------------------------------------------
>>> -----------------
>>> This message and any files or text attached to it are intended only for
>>> the recipients named above, and contain information that may be
>>> confidential or privileged. If you are not an intended recipient, or 
>>> you
>>> have reason to believe that you received this message in error, you 
>>> must
>>> not read, copy, use or disclose this communication. Please also notify
>>> the sender by replying to this message, and then delete all copies 
>>> of it
>>> from your system. Thank you.
>>>
>>>
>>>
>>>
>>>
>>>     
>>
>>   
>
>






More information about the Mondrian mailing list