[Mondrian] caching problem in latest Mondrian code

John V. Sichi jsichi at gmail.com
Mon Apr 7 04:29:12 EDT 2008


Thanks Will.  We'll give the perf tests another run once the deadlock 
fix is available.

JVS

Will Gorman wrote:
> I've checked in a fix for the caching inconsistency.  I'm still
> investigating the deadlock issue, the fix for the deadlock might require
> a little bit more effort.  I'm still attempting to create a test case
> that demonstrates the problem.
> 
> I appreciate you taking the time to help find these issues.  I've
> updated one of the unit tests to detect the correct cache hits at all
> three layers of the cache.  I've also updated the "todo" documentation
> with details on the cache object.  Please also read the Caching Strategy
> description at the beginning of the RolapCubeHierarchyMemberReader inner
> class for additional information.
> 
> Will
> 
> On Sat, 2008-04-05 at 00:40 -0700, John V. Sichi wrote:
>> After hitting the deadlock, we went ahead with analysis on the
>> single-thread portion of the LucidEra perf test suite, which had
>> completed before the concurrency portion where the deadlock occurred.
>>
>> The results showed significant degradation for just about every report,
>> so I poked around with some of the unit tests.  The MDX for TestCase
>> "testGrandTotals" in
>> testsrc/main/mondrian/test/clearview/GrandTotalTest.ref.xml exhibits the
>> problem (takes 23 seconds to run, compared to 18 seconds for our old
>> stable version).  I turned on SQL tracing, and saw a lot more SQL than
>> in the old version.  In particular, I saw the 'Drink' query below over
>> and over, which made me think the member caching is broken.
>>
>> Here's what I see happening while tracing.
>> SmartMemberReader.getMemberChildren first calls
>> cacheHelper.getChildrenFromCache, and doesn't get anything.  Further on,
>> it calls readMemberChildren, which is supposed to call
>> cacheHelper.putChildren to store them.  However, readMemberChildren is
>> overridden by subclass
>> RolapCubeHierarchy.RolapCubeHierarchyMemberReader, and its
>> implementation does *not* call cacheHelper.putChildren at all!  Instead,
>> it calls into rolapCubeCacheHelper.putChildren, but rolapCubeCacheHelper
>> is a different object (there's a todo at the top of the class for
>> explaining the distinction).  So next time through,
>> SmartMemberReader.readMemberChildren is going to have to fetch them
>> again, over and over.
>>
>> BTW, there's a lot of copy-and-paste between the two implementations of
>> readMemberChildren; someone might want to do some refactoring.
>>
>> SQL for cache misses:
>>
>> select "product_class"."product_department" from "product_class" as
>> "product_class", "product" as "product", "sales_fact_1997" as
>> "sales_fact_1997" where "sales_fact_1997"."product_id" =
>> "product"."product_id" and "product"."product_class_id" =
>> "product_class"."product_class_id" and "product_class"."product_family"
>> = 'Drink' and ("product_class"."product_family" = 'Drink') group by
>> "product_class"."product_department" order by
>> "product_class"."product_department" ASC
>>
>> Relevant stack:
>>
>> 	at
>> mondrian.rolap.SqlMemberSource.getMemberChildren2(SqlMemberSource.java:721)
>> 	at
>> mondrian.rolap.SqlMemberSource.getMemberChildren(SqlMemberSource.java:650)
>> 	at
>> mondrian.rolap.SqlMemberSource.getMemberChildren(SqlMemberSource.java:625)
>> 	at
>> mondrian.rolap.RolapCubeHierarchy$RolapCubeHierarchyMemberReader.readMemberChildren(RolapCubeHierarchy.java:470)
>> 	at
>> mondrian.rolap.SmartMemberReader.getMemberChildren(SmartMemberReader.java:201)
>> 	at
>> mondrian.rolap.RolapSchemaReader.getMemberChildren(RolapSchemaReader.java:290)
>> 	at
>> mondrian.olap.DelegatingSchemaReader.getMemberChildren(DelegatingSchemaReader.java:194)
>> 	at
>> mondrian.olap.fun.DescendantsFunDef.descendantsByLevel(DescendantsFunDef.java:310)
>>
>>
>> JVS
>>
>>
>> _______________________________________________
>> Mondrian mailing list
>> Mondrian at pentaho.org
>> http://lists.pentaho.org/mailman/listinfo/mondrian
>>
> 
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
> 





More information about the Mondrian mailing list