[Mondrian] Inefficiencies with Lists

Eric McDermid mcdermid at stonecreek.com
Thu Oct 22 16:29:30 EDT 2009


I haven't investigated enough to know for sure it's related, but this  
sounds like something I've seen recently investigating slowdowns  
introduced from 2.4 to 3.1.1.

In particular, there's a serious inefficiency when an instance of one  
of the RolapNamedSetEvaluator anonymous Iterable classes gets handed  
to mondrian/olap.fun.FunUtil.count().  The implementation of count  
(which hasn't changed) checks to see if the iterable passed in is an  
instance of Collection.  If so, it returns the result of size(); if  
not, it iterates through the iterator each time to get a count.

Unfortunately, neither of the inner classes implements Collection,  
despite the fact that they both work internally from a List, so we get  
a lot of unnecessary iterations.  In 2.4, the object passed in was  
actually a List, so this wasn't an issue.

For reference, The query I'm looking at has ~30K items being counted,  
and FunUtil.count() executes 46 times.  In 2.4, that had a cumulative  
hit of less than half a second.  In 3.1.1, it's more like 77 seconds.

I think the right thing to do here is extend the internal classes of  
RolapNamedSetEvaluator to implement Collection (which itself extends  
Iterable) rather than Iterable directly.  Since it's implemented with  
a list internally, this should be pretty easy.

I'm working on fixing this today, and I'll point you to the changelist  
once I'm done in case you want to try it yourself.

  -- Eric

On Oct 22, 2009, at 9:25 AM, Matt Campbell wrote:

>
> The simple crossjoin query in the unit test below runs in about 99  
> seconds on my laptop.  After noticing a hotspot in JProfiler, I  
> tried setting the result style to ResultStyle.LIST and reran.  It  
> completed in 11 seconds.  I haven't investigated much yet, but I  
> thought I'd throw this observation out for feedback.  It looks like  
> the LinkedList is not being accessed sequentially, causing a  
> significant bottleneck in list retrieval.
>
>
>  public void testLargeResult() {
>          long start = System.currentTimeMillis();
>          Result result = executeQuery("select  
> {  crossjoin( customers.[city].members, " +
>                  "crossjoin( [store type].[store type].members,   
> product.[product name].members)) }" +
>                  " on 0 from sales");
>          System.out.println("elapsed time:  " +  
> (System.currentTimeMillis() - start));
>          System.out.println("size=" + result.getAxes() 
> [0].getPositions().size());
>      }
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20091022/22f9da31/attachment.html 


More information about the Mondrian mailing list