[Mondrian] per query memory limit?

Julian Hyde jhyde at pentaho.com
Wed Aug 7 13:40:13 EDT 2013


On Aug 7, 2013, at 10:26 AM, "Wright, Jeff (Truven Health)" <jeff.s.wright at truvenhealth.com<mailto:jeff.s.wright at truvenhealth.com>> wrote:

Revisiting this thread… We have spent some time on code changes and testing related to counting cell and member requests at the point of loading from the DBMS. I’m trying to figure out how to think about how this interacts with caching. Looking for some reactions…

The cell requests seem to be only for cells that are not already cached (I expect members are same). My thought experiment on this is that potentially a clever/lucky user could work around a cell request limit by carefully working up to the full query. For example, if my query is comparing 2012 data to 2011, and gets failed for too many cell requests, I could run a query first for 2011 and generate fewer cell requests. But that doesn’t really reduce the memory overhead of my combo query… or does it?

If you mean could someone game the system and steal resources from other users. Yes. But unless Mondrian actually does per-user resource accounting that's unavoidable. Per-user resource accounting would be so heavyweight that the chances are that everyone would lose.

My thinking so far is that this would actually be a combined limit, something like

If ( ( 2 * memberRequests + cellRequests ) > requestThreshold )
                throw new RequestLimitException();

But maybe we need to take into account the size of member and cell caches? I haven’t looked to see how accessible that is.

I think a hybrid limit would work OK.  I'd replace the 2 with 10, because members take a lot more space than cells (especially when the cells are dense).

The member and cell caches don't have a "size". The JVM has a memory limit, that's it.

You can submit this as a patch, but I'm not yet convinced that this should be in the main code base. I've not heard of anyone else with this requirement.

Julian

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20130807/5692f51a/attachment-0001.html 


More information about the Mondrian mailing list