[Mondrian] Odd behaviors found while profiling/tuning Mondrian
mcdermid at stonecreek.com
Wed Jan 27 11:38:37 EST 2010
OK, here's a puzzler...
I'm attempting to bring a Mondrian 2.4-based system forward to
Mondrian 3.1.1 (and eventually, top of tree). At this point, the main
stumbling block is a 40-50% degradation in "fully cached" performance
(i.e., after multiple runs of the same query, so that SQL execution
time isn't a factor).
One of the oddest things I'm seeing is that more than half of the
query execution time (and in some cases up to almost 70% of it) is
spent in java.lang.System.arraycopy(), invoked from the
columnPredicateList.toArray() call within
rolap.agg.CellRequest.check(). More specifically, I'm seeing the
amount of time spent in that method increase with later changelists,
yet the size of the ArrayList in question is constant (5 members) and
the toArray() method is invoked exactly the same number of times.
Given that arraycopy() does a shallow copy, the actual contents of the
array shouldn't matter.
One of the first places this jump occurs is with changelist 10203,
which introduced RolapCubeMember to deal with shared dimensions. I
know this change introduced some other memory and performance issues
(which have since been partially ameliorated), but I can't see how it
would affect this call.
Has anyone else seen similar results, or encountered arraycopy()
issues elsewhere? I'm sure I'm not the first to throw Mondrian on a
large dataset under a profiler.
To try to rule out a problem in the profiling tool, I've verified this
with two different profiling tools (JProfiler and YourKit) and gotten
consistent results. Of course, it is possible that both rely on the
same underlying JVM hooks (and so, if wrong, could both be wrong in
the same way).
For reference, this is running on JDK 1.5 vs MySQL (and on a Mac, not
that that should matter). Unfortunately, there are some other issues
with the surrounding application that prevent me from moving up to JDK
1.6 right now, or I'd have tried CPU profiling in VisualVM as well.
The fact and dimension table have approximately 890K rows each.
More information about the Mondrian