[Mondrian] Unnecessary sorting in axis loading?
jhyde at pentaho.com
Fri May 1 17:30:02 EDT 2009
> Is there any reason we need to execute sorts prior to building the
> RolapAxis, or does it make sense for us to look for ways to avoid it?
Definitely makes sense to try to avoid a sort.
> Obviously, the last sort needs to happen in order to produce
> the right
> results, but if the purpose of the first pass through the axes is to
> make sure all of the members are loaded, then sorting at that point
> seems like unnecessary overhead that we might be able to eliminate,
> since even without sorting all of them will get loaded.
We don't make such a clear-cut distinction between the 'loading pass' and
'final pass'. If all cells are already loaded, evaluation might take only 1
pass; if there are complex calculations, it may take more than 2. So, I
don't want to introduce a huge distinction between passes.
In particular, consider the case where a sort is important to semantics,
e.g. Head(Order(...), ...) or Head(Crossjoin(Order(...), Order(...)), ...).
Can you propose a way to detect that (a) we have had cache misses so this is
definitely not the final pass and (b) the order that this function returns
results is not necessary to ensure the correct pattern of cache accesses?
Also note that I fixed a problem relating to sorting this week. See change
12668, and RolapResult.mergeAxes. mergeAxes will sort repeatedly if there
are many columns, and when both axes have NONEMPTY, and some other side
conditions. It may not apply to the cases you are interested in.
More information about the Mondrian