[Mondrian] Changes to Mondrian's caching architecture

Joe Barnett thejoe at gmail.com
Wed Jan 18 11:43:56 EST 2012

Hi Julian,

This is very cool, look forward to playing around with it.  A few questions:

-Have you measured the performance impact?  I'd imagine there's a
(hopefully) negligible single-threaded overhead cost to the thread
communication and better overall performance/throughput under
concurrent load, but would be good to quantify that.  (I'll certainly
do the same when I have time to try out the new code)

-What is the lifecycle of the new "cache manager" and "worker"
threads, and what objects are they tied to?  (Is the "cache manager"
thread per connection/star/cube/global?)  Also, how do these interact
with/supersede the RolapResultShepherd implementation that Luc added
in the last release?  Relatedly, are you still opposed to adding any
hooks to set context for the connection around these thread/callable
implementations, or can you think of a way that you might find
something similar acceptable? (FWIW, my previous diff was actually
wrong, and it was the callable created in RolapConnection#execute()
that needed the wrapping, not the thread/pool creation itself)

-It sounds like these changes apply only to the cell/segment cache,
and not the member caches?  Are there any plans on implementing
similar changes (both the Actor model cache, and the pluggable caching
architecture) for the member caches (which I think are just in
SmartMemberReader/MemberCacheHelper, if I remember right)?


On Mon, Jan 16, 2012 at 9:48 AM, Julian Hyde <jhyde at pentaho.com> wrote:
> We have made minor changes to the SegmentCache SPI. These will be in the upcoming release. (We figured, better to fix them now.)
> 1. SegmentHeader.Column is now a top-level class, SegmentColumn.
> 2. SegmentCache, SegmentBody, SegmentHeader, SegmentColumn are now in mondrian.spi package.
> 3. Methods on SegmentCache that return futures (and were assumed to return immediately) now return regular results (and it's OK if it they take some time to return). For example, 'Future<SegmentBody> get(SegmentHeader)' is now 'SegmentBody get(SegmentHeader)'.
> 4. Mondrian used to timeout on cache requests, configurable using parameters. These parameters are now obsolete. If you want timeouts, configure your cache to do so.
> By the way, I was wrong about the release number. It will be release Mondrian version 3.4, not Mondrian version 3.3.1 as I said in my email and blog post.
> Julian
> On Jan 16, 2012, at 1:42 AM, Pedro Alves wrote:
>> Julian - Will this change the current cache interface in place? We have
>> a ongoing implementation for hazelcast (CDC, in http://is.gd/wlFwlW)
>> On 01/14/2012 11:54 PM, Julian Hyde wrote:
>>> Mondrian developers,
>>> A heads up that I checked in some architectural changes this week. First
>>> the executive summary:
>>> 1. Mondrian should do the same thing as it did before, but scale up
>>> better to more concurrent queries and more cores.
>>> 2. Since this is a fairly significant change in the architecture, I'd
>>> appreciate if you kicked the tires, to make sure I didn't break anything.
>>> Now the longer version. (So long, I'll probably duplicate as a blog post.)
>>> Since we introduced external caches in Mondrian 3.3, we were aware that
>>> we were putting a strain on the caching architecture. The caching
>>> architecture has needed modernization for a while, but external caches
>>> made it worse. First, a call to an external cache can take a significant
>>> amount of time: depending on the cache, it might do a network I/O, and
>>> so take several orders of magnitude longer than a memory access. Second,
>>> we introduced external caching and introduced in-cache rollup, and for
>>> both of these we had to beef up the in-memory indexes needed to organize
>>> the cache segments.
>>> Previously we'd used a critical section approach: any thread that wanted
>>> to access an object in the cache locked out the entire cache. As the
>>> cache data structures became more complex, those operations were taking
>>> longer. To improve scalability, we adopted a radically different
>>> architectural pattern, called the Actor Model. Basically, one thread,
>>> called the Cache Manager is dedicated to looking after the cache index.
>>> Any client thread that wants to find a segment in the cache, or to add a
>>> segment to the cache, or create a segment by rolling up existing
>>> segments, or flush the cache sends a message to the Cache Manager.
>>> Ironically, the cache manager does not get segments from external
>>> caches. As I said earlier, external cache accesses can take a while, and
>>> the cache manager is super-busy. The cache manager tells the client the
>>> segment key to ask the external cache for, and the client does the
>>> asking. When a client gets a segment, it stores it in its private
>>> storage (good for the duration of a query) so it doesn't need to ask the
>>> cache manager again. Since a segment can contain thousands of cells,
>>> even large queries typically only make a few requests to the cache manager.
>>> The external cache isn't just slow; it is also porous. It can have a
>>> segment one minute, and forget it the next. The Mondrian client thread
>>> that gets the cache miss will tell the cache manager to remove the
>>> segment from its index (so Mondrian doesn't ask for it again), and
>>> formulate an alternative strategy to find it. Maybe the required cell
>>> exists in another cached segment; maybe it can be obtained by rolling up
>>> other segments in cache (but they, too, could have gone missing without
>>> notice). If all else fails, we can generate SQL to populate the required
>>> segment from the database (a fact table, or if possible, an aggregate
>>> table).
>>> Since the cache manager is too busy to talk to the external cache, it is
>>> certainly too busy to execute SQL statements. From the cache manager's
>>> perspective, SQL queries take an eternity (several million CPU cycles
>>> each), so it farms out SQL queries to worker threads. The cache manager
>>> marks that segment as 'loading'. If another query thread asks the cache
>>> manager for a cell that would be in that segment, it receives
>>> java.util.concurrent.Future<SegmentBody> that will be populated as soon
>>> as the segment arrives. When that segment returns, the query thread
>>> pushes the segment into the cache, and tells the cache manager to update
>>> the state of that segment from 'loading' to 'ready'.
>>> The Actor Model is a radically different architecture. First, let's look
>>> at the benefits. Since one thread is managing an entire subsystem, you
>>> can just remove all locking. This is liberating. Within the subsystem,
>>> you can code things very simply, rather than perverting your data
>>> structures for thread-safety. You don't even need to use
>>> concurrency-safe data structures like CopyOnWriteArrayList, you can just
>>> use the fastest data structure that does the job. Once you remove
>>> concurrency controls such as 'synchronized' blocks, and access from only
>>> one thread, the data structure becomes miraculously faster. How can that
>>> be? The data structure now resides in the thread's cache, and when you
>>> removed the concurrency controls, you were also removing memory barriers
>>> that forced changes to be written through L1 and L2 cache to RAM, which
>>> is up to 200 times slower [see
>>> http://julianhyde.blogspot.com/2010/11/numbers-everyone-should-know.html ].
>>> Migrating to the Actor Model wasn't without its challenges. First of
>>> all, you need to decide which data structures and actions should be
>>> owned by the actor. I believe we got that one right. I found that most
>>> of the same things needed to be done, but by different threads than
>>> previously; so the task we mainly about moving code around. We needed to
>>> refine the data structures that were passed between "query", "cache
>>> manager" and "worker" threads, to make sure that they were immutable.
>>> If, for instance, you want the query thread to find other useful work to
>>> do while it is waiting for a segment, it shouldn't be modifying a data
>>> structure that it put into the cache manager's request queue. In a
>>> future blog post, I'll describe in more detail the challenges & benefits
>>> of migrating one component of a complex software system to the Actor Model.
>>> Not all caches are equal. Some, like JBoss Infinispan, are able to share
>>> cache items (in our case, segments containing cell values) between nodes
>>> in a cluster, and to use redundancy to ensure that cache items are never
>>> lost. Infinispan calls itself a "data grid", and I became convinced that
>>> it is genuinely a different kind of beast than a mere cache. To support
>>> data grids, we added hooks so that a cache can tell Mondrian about
>>> segments that have been added to other nodes in a cluster. This way,
>>> Mondrian becomes a genuine cluster. If I execute query X on node 1, it
>>> will put segments into the data grid that will make the query you are
>>> about to submit, query Y on node 2, execute faster.
>>> As you can tell by the enthusiastic length of this post, I am very
>>> excited about this change to Mondrian's architecture. Outwardly,
>>> Mondrian executes the same MDX queries the same as it ever did. But the
>>> internal engine can scale better when running on a modern CPU with many
>>> cores; due to the external caches, the cache behave much more
>>> predictably; and you can create clusters of Mondrian nodes that share
>>> their work and memory.
>>> The changes will be released soon as Mondrian version 3.3.1, but you can
>>> help by downloading from the main line (or from CI), kicking the tires,
>>> and letting us know if you find any problems.
>>> Julian
>>> _______________________________________________
>>> Mondrian mailing list
>>> Mondrian at pentaho.org
>>> http://lists.pentaho.org/mailman/listinfo/mondrian
>> _______________________________________________
>> Mondrian mailing list
>> Mondrian at pentaho.org
>> http://lists.pentaho.org/mailman/listinfo/mondrian
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian

More information about the Mondrian mailing list