[Mondrian] Changes to Mondrian's caching architecture

Julian Hyde jhyde at pentaho.com
Mon Jan 16 12:48:23 EST 2012


We have made minor changes to the SegmentCache SPI. These will be in the upcoming release. (We figured, better to fix them now.)

1. SegmentHeader.Column is now a top-level class, SegmentColumn.

2. SegmentCache, SegmentBody, SegmentHeader, SegmentColumn are now in mondrian.spi package.

3. Methods on SegmentCache that return futures (and were assumed to return immediately) now return regular results (and it's OK if it they take some time to return). For example, 'Future<SegmentBody> get(SegmentHeader)' is now 'SegmentBody get(SegmentHeader)'.

4. Mondrian used to timeout on cache requests, configurable using parameters. These parameters are now obsolete. If you want timeouts, configure your cache to do so.

By the way, I was wrong about the release number. It will be release Mondrian version 3.4, not Mondrian version 3.3.1 as I said in my email and blog post.

Julian

On Jan 16, 2012, at 1:42 AM, Pedro Alves wrote:

> 
> 
> Julian - Will this change the current cache interface in place? We have 
> a ongoing implementation for hazelcast (CDC, in http://is.gd/wlFwlW)
> 
> On 01/14/2012 11:54 PM, Julian Hyde wrote:
>> Mondrian developers,
>> 
>> A heads up that I checked in some architectural changes this week. First
>> the executive summary:
>> 
>> 1. Mondrian should do the same thing as it did before, but scale up
>> better to more concurrent queries and more cores.
>> 
>> 2. Since this is a fairly significant change in the architecture, I'd
>> appreciate if you kicked the tires, to make sure I didn't break anything.
>> 
>> Now the longer version. (So long, I'll probably duplicate as a blog post.)
>> 
>> Since we introduced external caches in Mondrian 3.3, we were aware that
>> we were putting a strain on the caching architecture. The caching
>> architecture has needed modernization for a while, but external caches
>> made it worse. First, a call to an external cache can take a significant
>> amount of time: depending on the cache, it might do a network I/O, and
>> so take several orders of magnitude longer than a memory access. Second,
>> we introduced external caching and introduced in-cache rollup, and for
>> both of these we had to beef up the in-memory indexes needed to organize
>> the cache segments.
>> 
>> Previously we'd used a critical section approach: any thread that wanted
>> to access an object in the cache locked out the entire cache. As the
>> cache data structures became more complex, those operations were taking
>> longer. To improve scalability, we adopted a radically different
>> architectural pattern, called the Actor Model. Basically, one thread,
>> called the Cache Manager is dedicated to looking after the cache index.
>> Any client thread that wants to find a segment in the cache, or to add a
>> segment to the cache, or create a segment by rolling up existing
>> segments, or flush the cache sends a message to the Cache Manager.
>> 
>> Ironically, the cache manager does not get segments from external
>> caches. As I said earlier, external cache accesses can take a while, and
>> the cache manager is super-busy. The cache manager tells the client the
>> segment key to ask the external cache for, and the client does the
>> asking. When a client gets a segment, it stores it in its private
>> storage (good for the duration of a query) so it doesn't need to ask the
>> cache manager again. Since a segment can contain thousands of cells,
>> even large queries typically only make a few requests to the cache manager.
>> 
>> The external cache isn't just slow; it is also porous. It can have a
>> segment one minute, and forget it the next. The Mondrian client thread
>> that gets the cache miss will tell the cache manager to remove the
>> segment from its index (so Mondrian doesn't ask for it again), and
>> formulate an alternative strategy to find it. Maybe the required cell
>> exists in another cached segment; maybe it can be obtained by rolling up
>> other segments in cache (but they, too, could have gone missing without
>> notice). If all else fails, we can generate SQL to populate the required
>> segment from the database (a fact table, or if possible, an aggregate
>> table).
>> 
>> Since the cache manager is too busy to talk to the external cache, it is
>> certainly too busy to execute SQL statements. From the cache manager's
>> perspective, SQL queries take an eternity (several million CPU cycles
>> each), so it farms out SQL queries to worker threads. The cache manager
>> marks that segment as 'loading'. If another query thread asks the cache
>> manager for a cell that would be in that segment, it receives
>> java.util.concurrent.Future<SegmentBody> that will be populated as soon
>> as the segment arrives. When that segment returns, the query thread
>> pushes the segment into the cache, and tells the cache manager to update
>> the state of that segment from 'loading' to 'ready'.
>> 
>> The Actor Model is a radically different architecture. First, let's look
>> at the benefits. Since one thread is managing an entire subsystem, you
>> can just remove all locking. This is liberating. Within the subsystem,
>> you can code things very simply, rather than perverting your data
>> structures for thread-safety. You don't even need to use
>> concurrency-safe data structures like CopyOnWriteArrayList, you can just
>> use the fastest data structure that does the job. Once you remove
>> concurrency controls such as 'synchronized' blocks, and access from only
>> one thread, the data structure becomes miraculously faster. How can that
>> be? The data structure now resides in the thread's cache, and when you
>> removed the concurrency controls, you were also removing memory barriers
>> that forced changes to be written through L1 and L2 cache to RAM, which
>> is up to 200 times slower [see
>> http://julianhyde.blogspot.com/2010/11/numbers-everyone-should-know.html ].
>> 
>> Migrating to the Actor Model wasn't without its challenges. First of
>> all, you need to decide which data structures and actions should be
>> owned by the actor. I believe we got that one right. I found that most
>> of the same things needed to be done, but by different threads than
>> previously; so the task we mainly about moving code around. We needed to
>> refine the data structures that were passed between "query", "cache
>> manager" and "worker" threads, to make sure that they were immutable.
>> If, for instance, you want the query thread to find other useful work to
>> do while it is waiting for a segment, it shouldn't be modifying a data
>> structure that it put into the cache manager's request queue. In a
>> future blog post, I'll describe in more detail the challenges & benefits
>> of migrating one component of a complex software system to the Actor Model.
>> 
>> Not all caches are equal. Some, like JBoss Infinispan, are able to share
>> cache items (in our case, segments containing cell values) between nodes
>> in a cluster, and to use redundancy to ensure that cache items are never
>> lost. Infinispan calls itself a "data grid", and I became convinced that
>> it is genuinely a different kind of beast than a mere cache. To support
>> data grids, we added hooks so that a cache can tell Mondrian about
>> segments that have been added to other nodes in a cluster. This way,
>> Mondrian becomes a genuine cluster. If I execute query X on node 1, it
>> will put segments into the data grid that will make the query you are
>> about to submit, query Y on node 2, execute faster.
>> 
>> As you can tell by the enthusiastic length of this post, I am very
>> excited about this change to Mondrian's architecture. Outwardly,
>> Mondrian executes the same MDX queries the same as it ever did. But the
>> internal engine can scale better when running on a modern CPU with many
>> cores; due to the external caches, the cache behave much more
>> predictably; and you can create clusters of Mondrian nodes that share
>> their work and memory.
>> 
>> The changes will be released soon as Mondrian version 3.3.1, but you can
>> help by downloading from the main line (or from CI), kicking the tires,
>> and letting us know if you find any problems.
>> 
>> Julian
>> 
>> 
>> 
>> _______________________________________________
>> Mondrian mailing list
>> Mondrian at pentaho.org
>> http://lists.pentaho.org/mailman/listinfo/mondrian
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian



More information about the Mondrian mailing list