[Mondrian] CellBatchSize

Luc Boudreau lucboudreau at gmail.com
Thu Jun 27 09:34:13 EDT 2013


We have a couple of comments in the code about what we were foreseeing in
the future. What we'd like to get at is a pluggable system to define the
batching rules, not just in terms of size, but also being able to determine
if it is worth sharing a particular batch across threads or would it be
cheaper to duplicate some cells. One concrete example of this is a big
query which pulls a lot of cells and takes a lot of time to execute, while
a second smaller query comes in afterwards and has to wait for a subset of
the big segment. Sometimes, it is cheaper and more effective to fragment
the cache.

I'd prefer that we address this issue in its broader application, rather
than focus solely on the number of cells.




On Thu, Jun 27, 2013 at 9:22 AM, Matt Campbell <mcampbell at pentaho.com>wrote:

> There have been reports in the forum over the past few months of cases
> where performance is much worse in Mondrian 3.5/6 compared to 3.3.  What I
> think is going on is that some queries significantly exceed the
> cellBatchSize, causing a whole sequence of segment load queries, each with
> a different IN list for the items in that particular batch.  The benefits
> of batching cells in these cases are greatly outweighed by the cost of
> extra SQL queries.****
>
> ** **
>
> A couple questions:****
>
> **1)      **I notice that the default value of cellBatchSize is -1, which
> I would interpret as meaning that there is no hard limit on the number of
> cells batched together.  In FastBatchingCellReader, though, if
> cellBatchSize is less than 0 we set the limit at a hardcoded 100000.
> Should we provide some way of truly having no hard limit for cellBatchSize?
> ****
>
> ** **
>
> **2)      **More generally--what is the benefit of batching, and what can
> we do to balance that against the cost of extra queries?****
>
> ** **
>
> ** **
>
> ** **
>
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20130627/62b4d85a/attachment.html 


More information about the Mondrian mailing list