[Mondrian] Eigenbase perforce change 14818 for review
Julian Hyde
jhyde at pentaho.com
Fri Dec 2 17:08:32 EST 2011
Luc,
Do you have any evidence that there was a problem with CellKey? It is already very well optimized for memory. (Better than SegmentCellKey, in the case of 0, 1 or 2 dimensions, which are fairly common, because it uses inline members, rather than an allocated array.)
In SegmentCellKey you state that it being a "top level class ... prevents it from having a reference to its parent class and makes it easier for GC". But note that since CellKey is an interface, its inner classes Zero, One etc. static; they have no pointer to a parent object.
It's true that CellKey is mutable. For these purposes, I'd prefer it wasn't. SegmentCellKey.getOrdinals() returns the naked array, so it's not totally immutable either.
If there are no clear advantages over CellKey, I suggest we stick with CellKey for everything.
SegmentBuilder.intersect was having issues when one of the arguments was the empty set. I have fixed it, and will check in shortly.
Julian
On Dec 2, 2011, at 11:58 AM, Luc Boudreau wrote:
> http://p4web.eigenbase.org/@md=d&c=6PU@/14818?ac=10
>
> Change 14818 by lucboudreau at luc-mondrian-3.2 on 2011/12/02 11:56:44
>
> MONDRIAN-PACINO:
>
> Optimization of SegmentBuilder.rollup(). It now picks the best dataset implementation to use and enforces the rules concerning sparse vs. dense datasets.
>
> Fixes failing tests due to SegmentCacheIndex which was not considering compound predicates for rollup queries.
>
> Extends Aggregator so that it can aggregate raw datasets rather than only being able to aggregate compiled expressions.
>
> Makes SegmentBuilder.rollup() use the measure's aggregator to perform the rollup calculation.
>
> Creates SegmentCellKey in mondrian.spi.
>
> Refactors SegmentCacheManager into an isntance of AggregationManager.
>
> Adds cleanup code for SegmentCacheManager and its resources.
>
> Fixes a cast bug in MockSegmentCache.
>
> Unit tests for SegmentBuilder.rollup() will come in the next checkin, after optimized versions of rollup() are created.
>
> Affected files ...
>
> ... //open/mondrian-release/pacino/src/main/mondrian/olap/Aggregator.java#2 edit
> ... //open/mondrian-release/pacino/src/main/mondrian/rolap/FastBatchingCellReader.java#12 edit
> ... //open/mondrian-release/pacino/src/main/mondrian/rolap/RolapAggregator.java#2 edit
> ... //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/AbstractSegmentBody.java#5 edit
> ... //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/AggregationManager.java#11 edit
> ... //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentBuilder.java#10 edit
> ... //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentCacheManager.java#12 edit
> ... //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentLoader.java#10 edit
> ... //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SparseSegmentBody.java#5 edit
> ... //open/mondrian-release/pacino/src/main/mondrian/rolap/cache/SegmentCacheIndexImpl.java#13 edit
> ... //open/mondrian-release/pacino/src/main/mondrian/spi/SegmentBody.java#4 edit
> ... //open/mondrian-release/pacino/src/main/mondrian/spi/SegmentCellKey.java#1 add
> ... //open/mondrian-release/pacino/testsrc/main/mondrian/rolap/agg/MockSegmentCache.java#7 edit
>
> Differences ...
>
> ==== //open/mondrian-release/pacino/src/main/mondrian/olap/Aggregator.java#2 (ktext) ====
>
> 2c2
> < // $Id: //open/mondrian-release/pacino/src/main/mondrian/olap/Aggregator.java#1 $
> ---
>> // $Id: //open/mondrian-release/pacino/src/main/mondrian/olap/Aggregator.java#2 $
> 13a14,17
>> import mondrian.spi.Dialect.Datatype;
>> import mondrian.spi.SegmentBody;
>>
>> import java.util.List;
> 23c27
> < * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/olap/Aggregator.java#1 $
> ---
>> * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/olap/Aggregator.java#2 $
> 39a44,72
>>
>> /**
>> * Tells Mondrian if this aggregator can perform fast aggregation
>> * using only the raw data of a given object type. This will
>> * determine if Mondrian will attempt to perform in-memory rollups
>> * on raw segment data by invoking {@link Aggregator#aggregate(Object[])}.
>> *
>> * <p>This is only invoked for rollup operations.
>> *
>> * @param datatype The datatype of the object we would like to rollup.
>> * @return True or false, depending on the support status.
>> */
>> boolean supportsFastAggregates(Datatype datatype);
>>
>> /**
>> * Applies this aggregator over a raw list of objects for a rollup
>> * operation. This is useful when the values are already resolved
>> * and we are dealing with a raw {@link SegmentBody} object.
>> *
>> * <p>Only gets called if {@link Aggregator#supportsFastAggregates()}
>> * is true.
>> *
>> * <p>This is only invoked for rollup operations.
>> *
>> * @param rawData An array of values in its raw form, to be aggregated.
>> * @return A rolled up value of the raw data.
>> * if the object type is not supported.
>> */
>> Object aggregate(List<Object> rawData);
>
> ==== //open/mondrian-release/pacino/src/main/mondrian/rolap/FastBatchingCellReader.java#12 (ktext) ====
>
> 2c2
> < // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/FastBatchingCellReader.java#11 $
> ---
>> // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/FastBatchingCellReader.java#12 $
> 196,201c196,211
> < // Try to roll up.
> < Map<SegmentHeader, SegmentBody> rollup =
> < locateRollupCandidates(
> < request,
> < request.getMappedCellValues(),
> < key);
> ---
>> // Try to roll up if the measure's rollup aggregator supports
>> // "fast" aggregation from raw objects.
>> final Map<SegmentHeader, SegmentBody> rollup;
>> if (request.getMeasure().getAggregator().getRollup()
>> .supportsFastAggregates(request.getMeasure().getDatatype()))
>> {
>> // Don't even bother doing a segment lookup if we can't
>> // rollup that measure.
>> rollup =
>> locateRollupCandidates(
>> request,
>> request.getMappedCellValues(),
>> key);
>> } else {
>> rollup = null;
>> }
> 211c221,222
> < request.getConstrainedColumnsBitKey());
> ---
>> request.getConstrainedColumnsBitKey(),
>> request.getMeasure().getAggregator().getRollup());
>
> ==== //open/mondrian-release/pacino/src/main/mondrian/rolap/RolapAggregator.java#2 (ktext) ====
>
> 2c2
> < // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/RolapAggregator.java#1 $
> ---
>> // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/RolapAggregator.java#2 $
> 16a17,19
>> import mondrian.spi.Dialect;
>>
>> import java.util.List;
> 23c26
> < * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/RolapAggregator.java#1 $
> ---
>> * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/RolapAggregator.java#2 $
> 37a41,74
>> public boolean supportsFastAggregates(Dialect.Datatype dataType) {
>> switch (dataType) {
>> case Integer:
>> case Numeric:
>> return true;
>> default:
>> return false;
>> }
>> };
>> public Object aggregate(List<Object> rawData) {
>> assert rawData.size() > 0;
>> if (rawData.get(0) instanceof Integer) {
>> int totalValue = 0;
>> for (Object data : rawData) {
>> totalValue += (Integer)data;
>> }
>> return totalValue;
>> }
>> if (rawData.get(0) instanceof Double) {
>> double totalValue = 0d;
>> for (Object data : rawData) {
>> totalValue += (Double)data;
>> }
>> return totalValue;
>> }
>> if (rawData.get(0) instanceof Long) {
>> long totalValue = 0l;
>> for (Object data : rawData) {
>> totalValue += (Long)data;
>> }
>> return totalValue;
>> }
>> throw new IllegalArgumentException();
>> }
> 59a97,130
>> public boolean supportsFastAggregates(Dialect.Datatype dataType) {
>> switch (dataType) {
>> case Integer:
>> case Numeric:
>> return true;
>> default:
>> return false;
>> }
>> };
>> public Object aggregate(List<Object> rawData) {
>> assert rawData.size() > 0;
>> if (rawData.get(0) instanceof Integer) {
>> int min = Integer.MAX_VALUE;
>> for (Object data : rawData) {
>> min = Math.min(min, (Integer)data);
>> }
>> return min;
>> }
>> if (rawData.get(0) instanceof Double) {
>> double min = Double.MAX_VALUE;
>> for (Object data : rawData) {
>> min = Math.min(min, (Double)data);
>> }
>> return min;
>> }
>> if (rawData.get(0) instanceof Long) {
>> long min = Long.MAX_VALUE;
>> for (Object data : rawData) {
>> min = Math.min(min, (Long)data);
>> }
>> return min;
>> }
>> throw new IllegalArgumentException();
>> }
> 68a140,173
>> public boolean supportsFastAggregates(Dialect.Datatype dataType) {
>> switch (dataType) {
>> case Integer:
>> case Numeric:
>> return true;
>> default:
>> return false;
>> }
>> };
>> public Object aggregate(List<Object> rawData) {
>> assert rawData.size() > 0;
>> if (rawData.get(0) instanceof Integer) {
>> int min = Integer.MIN_VALUE;
>> for (Object data : rawData) {
>> min = Math.max(min, (Integer)data);
>> }
>> return min;
>> }
>> if (rawData.get(0) instanceof Double) {
>> double min = Double.MIN_VALUE;
>> for (Object data : rawData) {
>> min = Math.max(min, (Double)data);
>> }
>> return min;
>> }
>> if (rawData.get(0) instanceof Long) {
>> long min = Long.MIN_VALUE;
>> for (Object data : rawData) {
>> min = Math.max(min, (Long)data);
>> }
>> return min;
>> }
>> throw new IllegalArgumentException();
>> }
> 279a385,395
>>
>> /**
>> * By default, fast rollup is not supported for all classes.
>> */
>> public boolean supportsFastAggregates(Dialect.Datatype dataType) {
>> return false;
>> }
>>
>> public Object aggregate(List<Object> rawData) {
>> throw new UnsupportedOperationException();
>> }
>
> ==== //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/AbstractSegmentBody.java#5 (ktext) ====
>
> 2c2
> < // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/AbstractSegmentBody.java#4 $
> ---
>> // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/AbstractSegmentBody.java#5 $
> 12d11
> < import mondrian.rolap.CellKey;
> 13a13
>> import mondrian.spi.SegmentCellKey;
> 22c22
> < * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/AbstractSegmentBody.java#4 $
> ---
>> * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/AbstractSegmentBody.java#5 $
> 52,56c52,56
> < public Map<CellKey, Object> getValueMap() {
> < return new AbstractMap<CellKey, Object>() {
> < public Set<Entry<CellKey, Object>> entrySet() {
> < return new AbstractSet<Entry<CellKey, Object>>() {
> < public Iterator<Entry<CellKey, Object>> iterator() {
> ---
>> public Map<SegmentCellKey, Object> getValueMap() {
>> return new AbstractMap<SegmentCellKey, Object>() {
>> public Set<Entry<SegmentCellKey, Object>> entrySet() {
>> return new AbstractSet<Entry<SegmentCellKey, Object>>() {
>> public Iterator<Entry<SegmentCellKey, Object>> iterator() {
> 87c87
> < implements Iterator<Map.Entry<CellKey, Object>>
> ---
>> implements Iterator<Map.Entry<SegmentCellKey, Object>>
> 105,107c105,107
> < public Map.Entry<CellKey, Object> next() {
> < Pair<CellKey, Object> o =
> < Pair.of(CellKey.Generator.newCellKey(ordinals), next);
> ---
>> public Map.Entry<SegmentCellKey, Object> next() {
>> Pair<SegmentCellKey, Object> o =
>> Pair.of(new SegmentCellKey(ordinals), next);
>
> ==== //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/AggregationManager.java#11 (ktext) ====
>
> 2c2
> < // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/AggregationManager.java#10 $
> ---
>> // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/AggregationManager.java#11 $
> 38c38
> < * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/AggregationManager.java#10 $
> ---
>> * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/AggregationManager.java#11 $
> 57c57
> < public final Executor sqlExecutor =
> ---
>> public final ExecutorService sqlExecutor =
> 59c59
> < 10, 3, 1, 10, "mondrian.rolap.agg.AggregationManager$sqlExecutor");
> ---
>> 10, 0, 1, 10, "mondrian.rolap.agg.AggregationManager$sqlExecutor");
> 496a497,498
>> sqlExecutor.shutdown();
>> cacheMgr.shutdown();
>
> ==== //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentBuilder.java#10 (ktext) ====
>
> 2c2
> < // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentBuilder.java#9 $
> ---
>> // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentBuilder.java#10 $
> 11a12
>> import mondrian.olap.Aggregator;
> 20a22
>> import java.util.Map.Entry;
> 28c30
> < * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentBuilder.java#9 $
> ---
>> * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentBuilder.java#10 $
> 85c87,88
> < new SparseSegmentDataset(body.getValueMap());
> ---
>> new SparseSegmentDataset(
>> ((SparseSegmentBody)body).getCellKeyValueMap());
> 162a166,170
>> * @param keepColumns A list of column names to keep as part of
>> * the rolled up segment.
>> * @param targetBitkey The column bit key to match with the
>> * resulting segment.
>> * @param rollupAggregator The aggregator to use to rollup.
> 168c176,177
> < BitKey targetBitkey)
> ---
>> BitKey targetBitkey,
>> Aggregator rollupAggregator)
> 244,248c253,254
> < final Map<CellKey, Object> cellValues =
> < new HashMap<CellKey, Object>();
> < final int[] pos = new int[axes.length];
> < final Comparable[][] valueArrays =
> < new Comparable[firstHeader.getConstrainedColumns().length][];
> ---
>> final Map<SegmentCellKey, List<Object>> cellValues =
>> new HashMap<SegmentCellKey, List<Object>>();
> 249a256,258
>> final int[] pos = new int[axes.length];
>> final Comparable<?>[][] valueArrays =
>> new Comparable[firstHeader.getConstrainedColumns().length][];
> 256,258c265,266
> < valueArrays[z] =
> < keepColumns.contains(
> < firstHeader.getConstrainedColumns()[z].columnExpression)
> ---
>> valueArrays[z] = keepColumns.contains(
>> firstHeader.getConstrainedColumns()[z].columnExpression)
> 263,266c271,272
> < Map<CellKey, Object> v = body.getValueMap();
> < for (Map.Entry<CellKey, Object> vEntry : v.entrySet()) {
> < final CellKey cellKey = vEntry.getKey();
> < int[] ordinals = cellKey.getOrdinals();
> ---
>> Map<SegmentCellKey, Object> v = body.getValueMap();
>> for (Map.Entry<SegmentCellKey, Object> vEntry : v.entrySet()) {
> 268,269c274,275
> < for (int i = 0; i < ordinals.length; i++) {
> < final Comparable[] valueArray = valueArrays[i];
> ---
>> for (int i = 0; i < vEntry.getKey().getArrity(); i++) {
>> final Comparable<?>[] valueArray = valueArrays[i];
> 273,274c279,280
> < final int ordinal = ordinals[i];
> < final Comparable value = valueArray[ordinal];
> ---
>> final int ordinal = vEntry.getKey().getOrdinals()[i];
>> final Comparable<?> value = valueArray[ordinal];
> 280,281c286,289
> < Arrays.binarySearch(
> < axes[z].values, value);
> ---
>> Util.binarySearch(
>> axes[z].values,
>> 0, axes[z].values.length,
>> value);
> 285,293c293,295
> < final CellKey targetCellKey =
> < CellKey.Generator.newCellKey(pos);
> < final Object cellValue = vEntry.getValue();
> < final Object prevValue =
> < cellValues.put(targetCellKey, cellValue);
> < if (prevValue != null) {
> < cellValues.put(
> < targetCellKey,
> < plus(prevValue, cellValue));
> ---
>> final SegmentCellKey ck = new SegmentCellKey(pos);
>> if (!cellValues.containsKey(ck)) {
>> cellValues.put(ck, new ArrayList<Object>());
> 294a297
>> cellValues.get(ck).add(vEntry.getValue());
> 298c301
> < // Create body.
> ---
>> // Build the axis list.
> 301c304,306
> < for (AxisInfo axis : axes) {
> ---
>> final BitSet nullIndicators = new BitSet(axes.length);
>> int nbValues = 1;
>> for (int i = 0; i < axes.length; i++) {
> 304c309,396
> < axis.valueSet, axis.hasNull));
> ---
>> axes[i].valueSet, axes[i].hasNull));
>> nullIndicators.set(i, axes[i].hasNull);
>> nbValues *= axes[i].values.length;
>> }
>> final int[] axisMultipliers =
>> computeAxisMultipliers(axisList);
>>
>> final SegmentBody body;
>> // Peak at the values and determine the best way to store them
>> // (whether to use a dense native dataset or a sparse one.
>> if (cellValues.size() == 0) {
>> // Just store the data into an empty dense object dataset.
>> body =
>> new DenseObjectSegmentBody(
>> new Object[0],
>> axisList);
>> } else if (SegmentLoader.useSparse(
>> cellValues.size(),
>> cellValues.size() - nullIndicators.cardinality()))
>> {
>> // The rule says we must use a sparse dataset.
>> // First, aggregate the values of each key.
>> final Map<CellKey, Object> data =
>> new HashMap<CellKey, Object>();
>> for (Entry<SegmentCellKey, List<Object>> entry
>> : cellValues.entrySet())
>> {
>> data.put(
>> CellKey.Generator.newCellKey(entry.getKey().getOrdinals()),
>> rollupAggregator.aggregate(entry.getValue()));
>> }
>> body =
>> new SparseSegmentBody(
>> data,
>> axisList);
>> } else {
>> // Peek at the value class. We will use a native dataset
>> // if possible.
>> final Object peek =
>> cellValues.entrySet().iterator().next().getValue().get(0);
>> if (peek instanceof Double) {
>> final double[] data = new double[nbValues];
>> for (Entry<SegmentCellKey, List<Object>> entry
>> : cellValues.entrySet())
>> {
>> final int offset =
>> CellKey.Generator.getOffset(
>> entry.getKey().getOrdinals(), axisMultipliers);
>> data[offset] =
>> (Double)rollupAggregator.aggregate(entry.getValue());
>> }
>> body =
>> new DenseDoubleSegmentBody(
>> nullIndicators,
>> data,
>> axisList);
>> } else if (peek instanceof Integer) {
>> final int[] data = new int[cellValues.size()];
>> for (Entry<SegmentCellKey, List<Object>> entry
>> : cellValues.entrySet())
>> {
>> final int offset =
>> CellKey.Generator.getOffset(
>> entry.getKey().getOrdinals(), axisMultipliers);
>> data[offset] =
>> (Integer)rollupAggregator.aggregate(entry.getValue());
>> }
>> body =
>> new DenseIntSegmentBody(
>> nullIndicators,
>> data,
>> axisList);
>> } else {
>> final Object[] data = new Object[cellValues.size()];
>> for (Entry<SegmentCellKey, List<Object>> entry
>> : cellValues.entrySet())
>> {
>> final int offset =
>> CellKey.Generator.getOffset(
>> entry.getKey().getOrdinals(), axisMultipliers);
>> data[offset] =
>> (Object)rollupAggregator.aggregate(entry.getValue());
>> }
>> body =
>> new DenseObjectSegmentBody(
>> data,
>> axisList);
>> }
> 306,309d397
> < SegmentBody body =
> < new SparseSegmentBody(
> < cellValues,
> < axisList);
> 323c411
> < SegmentHeader header =
> ---
>> final SegmentHeader header =
> 338,343c426,453
> < private static Object plus(Object value1, Object value2) {
> < if (value1 instanceof Integer) {
> < return (Integer) value1 + (Integer) value2;
> < } else {
> < return (Double) value1 + (Double) value2;
> < }
> ---
>> /*
>> * TODO: Factor this out into ArraySortedSet or Util.
>> */
>> private static SortedSet<Comparable<?>> intersect(
>> SortedSet<Comparable<?>> set1,
>> SortedSet<Comparable<?>> set2)
>> {
>> final Iterator<Comparable<?>> it1 = set1.iterator();
>> final Iterator<Comparable<?>> it2 = set2.iterator();
>> final Comparable<?>[] result =
>> new Comparable[Math.max(set1.size(), set2.size())];
>> int i = 0;
>> Comparable e1 = it1.next();
>> Comparable e2 = it2.next();
>> while (e1 != null && e2 != null) {
>> final int compare = e1.compareTo(e2);
>> if (compare == 0) {
>> result[i++] = e1;
>> i++;
>> e1 = it1.next();
>> e2 = it2.next();
>> } else if (compare == 1) {
>> e2 = it2.next();
>> } else {
>> e1 = it1.next();
>> }
>> };
>> return new ArraySortedSet(result, 0, i);
> 346,349c456,457
> <
> < private static <E> SortedSet<E> intersect(
> < SortedSet<E> set1,
> < SortedSet<E> set2)
> ---
>> private static int[] computeAxisMultipliers(
>> List<Pair<SortedSet<Comparable<?>>, Boolean>> axes)
> 351,356c459,465
> < // TODO: There is a more efficient algorithm to intersect sets, given
> < // that they are sorted. Or use Comparable[] rather than SortedSet if
> < // more efficient.
> < final TreeSet<E> set = new TreeSet<E>(set1);
> < set.retainAll(set2);
> < return set;
> ---
>> final int[] axisMultipliers = new int[axes.size()];
>> int multiplier = 1;
>> for (int i = axes.size() - 1; i >= 0; --i) {
>> axisMultipliers[i] = multiplier;
>> multiplier *= axes.get(i).left.size();
>> }
>> return axisMultipliers;
> 358d466
> <
>
> ==== //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentCacheManager.java#12 (ktext) ====
>
> 2c2
> < // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentCacheManager.java#11 $
> ---
>> // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentCacheManager.java#12 $
> 243c243
> < * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentCacheManager.java#11 $
> ---
>> * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentCacheManager.java#12 $
> 246a247,249
>> private final Actor ACTOR;
>> public final SegmentCacheIndex segmentIndex;
>> private final Thread thread;
> 248,250c251
> < private static final Actor ACTOR;
> <
> < static {
> ---
>> public SegmentCacheManager() {
> 252,268c253,254
> < }
> <
> < public final SegmentCacheIndex segmentIndex =
> < new SegmentCacheIndexImpl(ACTOR.thread);
> <
> < static {
> < // Create and start thread for actor.
> < //
> < // Actor is shared between all servers. This reduces concurrency.
> < // This might become a concern for those executing several active
> < // servers in the same JVM.
> < // We tried creating one actor (and therefore thread) per server, but
> < // some applications (and in particular some tests) create lots of
> < // servers.
> < //
> < // The actor is shut down with the JVM.
> < final Thread thread = new Thread(ACTOR, "Mondrian segment cache");
> ---
>> thread = new Thread(
>> ACTOR, "mondrian.rolap.agg.SegmentCacheManager$ACTOR");
> 269a256
>> segmentIndex = new SegmentCacheIndexImpl(thread);
> 377a365,371
>> * Shuts down this cache manager and all active threads and indexes.
>> */
>> public void shutdown() {
>> execute(new ShutdownCommand());
>> }
>>
>> /**
> 662,666d655
> < /**
> < * Current thread. Not null if and only if actor is running. Can be used
> < * to check that data structures are called from the dedicated thread.
> < */
> < private Thread thread;
> 676d664
> < thread = Thread.currentThread();
> 716,717d703
> < } finally {
> < thread = null;
>
> ==== //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentLoader.java#10 (ktext) ====
>
> 2c2
> < // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentLoader.java#9 $
> ---
>> // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentLoader.java#10 $
> 44c44
> < * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentLoader.java#9 $
> ---
>> * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SegmentLoader.java#10 $
> 805c805
> < private static boolean useSparse(
> ---
>> static boolean useSparse(
>
> ==== //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SparseSegmentBody.java#5 (ktext) ====
>
> 2c2
> < // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SparseSegmentBody.java#4 $
> ---
>> // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SparseSegmentBody.java#5 $
> 12a13
>> import mondrian.spi.SegmentCellKey;
> 22c23
> < * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SparseSegmentBody.java#4 $
> ---
>> * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/agg/SparseSegmentBody.java#5 $
> 27a29
>> final SegmentCellKey[] intCellKeys;
> 36a39
>> this.intCellKeys = new SegmentCellKey[dataToSave.size()];
> 40a44
>> intCellKeys[i] = new SegmentCellKey(entry.getKey().getOrdinals());
> 55c59
> < public Map<CellKey, Object> getValueMap() {
> ---
>> public Map<CellKey, Object> getCellKeyValueMap() {
> 62a67,76
>>
>> @Override
>> public Map<SegmentCellKey, Object> getValueMap() {
>> final Map<SegmentCellKey, Object> map =
>> new HashMap<SegmentCellKey, Object>(keys.length * 3 / 2);
>> for (int i = 0; i < intCellKeys.length; i++) {
>> map.put(intCellKeys[i], data[i]);
>> }
>> return map;
>> }
>
> ==== //open/mondrian-release/pacino/src/main/mondrian/rolap/cache/SegmentCacheIndexImpl.java#13 (ktext) ====
>
> 2c2
> < // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/cache/SegmentCacheIndexImpl.java#12 $
> ---
>> // $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/cache/SegmentCacheIndexImpl.java#13 $
> 28c28
> < * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/cache/SegmentCacheIndexImpl.java#12 $
> ---
>> * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/rolap/cache/SegmentCacheIndexImpl.java#13 $
> 33a34,37
>> /**
>> * The fact map allows us to spot quickly which
>> * segments have facts relating to a given header.
>> */
> 35a40,50
>> /**
>> * The fuzzy fact map allows us to spot quickly which
>> * segments have facts relating to a given header, but doesn't
>> * consider the compound predicates in the key. This allows
>> * flush operations to be consistent.
>> */
>> // TODO Get rid of the fuzzy map once we have a way to parse
>> // compound predicates into rich objects that can be serialized
>> // as part of the SegmentHeader.
>> private final Map<List, FactInfo> fuzzyFactMap =
>> new HashMap<List, FactInfo>();
> 68c83,84
> < measureName);
> ---
>> measureName,
>> compoundPredicates);
> 104a121,129
>>
>> final List fuzzyFactKey = makeFuzzyFactKey(header);
>> FactInfo fuzzyFactInfo = fuzzyFactMap.get(fuzzyFactKey);
>> if (fuzzyFactInfo == null) {
>> fuzzyFactInfo = new FactInfo();
>> fuzzyFactMap.put(fuzzyFactKey, fuzzyFactInfo);
>> }
>> fuzzyFactInfo.headerList.add(header);
>> fuzzyFactInfo.bitkeyPoset.add(header.getConstrainedColumnsBitKey());
> 114a140,146
>> final List fuzzyFactKey = makeFuzzyFactKey(header);
>> final FactInfo fuzzyFactInfo = fuzzyFactMap.get(fuzzyFactKey);
>> fuzzyFactInfo.headerList.remove(header);
>> if (fuzzyFactInfo.headerList.size() == 0) {
>> fuzzyFactMap.remove(fuzzyFactKey);
>> }
>>
> 174c206
> < final List factKey = makeFactKey(
> ---
>> final List factKey = makeFuzzyFactKey(
> 180c212
> < final FactInfo factInfo = factMap.get(factKey);
> ---
>> final FactInfo factInfo = fuzzyFactMap.get(factKey);
> 247c279,280
> < header.measureName);
> ---
>> header.measureName,
>> header.compoundPredicates);
> 256c289,290
> < String measureName)
> ---
>> String measureName,
>> String[] compoundPredicates)
> 264c298,299
> < measureName);
> ---
>> measureName,
>> Arrays.deepToString(compoundPredicates));
> 272a308,334
>> header.measureName,
>> header.compoundPredicates);
>> }
>>
>> private List makeFactKey(
>> String schemaName,
>> ByteString schemaChecksum,
>> String cubeName,
>> String rolapStarFactTableName,
>> String measureName,
>> String[] compoundPredicates)
>> {
>> return Arrays.asList(
>> schemaName,
>> schemaChecksum,
>> cubeName,
>> rolapStarFactTableName,
>> measureName,
>> Arrays.deepToString(compoundPredicates));
>> }
>>
>> private List makeFuzzyFactKey(SegmentHeader header) {
>> return makeFuzzyFactKey(
>> header.schemaName,
>> header.schemaChecksum,
>> header.cubeName,
>> header.rolapStarFactTableName,
> 276c338
> < private List makeFactKey(
> ---
>> private List makeFuzzyFactKey(
> 306c368,369
> < measureName);
> ---
>> measureName,
>> compoundPredicates);
> 331c394,395
> < measureName);
> ---
>> measureName,
>> compoundPredicates);
>
> ==== //open/mondrian-release/pacino/src/main/mondrian/spi/SegmentBody.java#4 (ktext) ====
>
> 2c2
> < // $Id: //open/mondrian-release/pacino/src/main/mondrian/spi/SegmentBody.java#3 $
> ---
>> // $Id: //open/mondrian-release/pacino/src/main/mondrian/spi/SegmentBody.java#4 $
> 12d11
> < import mondrian.rolap.CellKey;
> 25c24
> < * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/spi/SegmentBody.java#3 $
> ---
>> * @version $Id: //open/mondrian-release/pacino/src/main/mondrian/spi/SegmentBody.java#4 $
> 34c33
> < Map<CellKey, Object> getValueMap();
> ---
>> Map<SegmentCellKey, Object> getValueMap();
>
> ==== //open/mondrian-release/pacino/testsrc/main/mondrian/rolap/agg/MockSegmentCache.java#7 (ktext) ====
>
> 2c2
> < // $Id: //open/mondrian-release/pacino/testsrc/main/mondrian/rolap/agg/MockSegmentCache.java#6 $
> ---
>> // $Id: //open/mondrian-release/pacino/testsrc/main/mondrian/rolap/agg/MockSegmentCache.java#7 $
> 28c28
> < * @version $Id: //open/mondrian-release/pacino/testsrc/main/mondrian/rolap/agg/MockSegmentCache.java#6 $
> ---
>> * @version $Id: //open/mondrian-release/pacino/testsrc/main/mondrian/rolap/agg/MockSegmentCache.java#7 $
> 65c65
> < SegmentColumn o = (SegmentColumn) ois.readObject();
> ---
>> SegmentHeader o = (SegmentHeader) ois.readObject();
More information about the Mondrian
mailing list