[Mondrian] RE: Eigenbase perforce change 13367 for review
Julian Hyde
jhyde at pentaho.com
Wed Feb 24 05:52:15 EST 2010
This is a smart fix, and I'm sure that it helps for really large schemas,
but is it always a win? We need RolapEvaluator.push() to be efficient, and
copying a treemap isn't that cheap: it involves an non-trivial iterator, and
lots of memory allocations. A map from integer to members can obviously be
implemented as an array. I want to be sure that we haven't made a critical
piece of code much more expensive in the usual just to improve algorithmic
complexity.
Julian
> -----Original Message-----
> From: Matt Campbell [mailto:Matthew.Campbell at thomson.com]
> Sent: Wednesday, February 03, 2010 1:28 PM
> To: Aaron Phillips; Ezequiel Cuellar; Julian Hyde; John V.
> Sichi; Will Gorman
> Subject: Eigenbase perforce change 13367 for review
>
> http://p4web.eigenbase.org/@md=d&c=6PU@//13367?ac=10
>
> Change 13367 by mkambol at guest_AA-5501 on 2010/02/03 13:26:50
>
> MONDRIAN: performance enhancment. RolapEvaluator now
> has a map that contains all members in the context minus any
> "all" members in context. The new list is used in
> getProperty and other places to significantly trim down the
> number of iterations needed for each cell. The amount of
> performance gain is proportional to the number of cells in
> the results times the number of dimensions in the schema.
>
> Affected files ...
>
> ...
> //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.j
> ava#2 edit
> ...
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
> gationManager.java#2 edit
> ...
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
> ator.java#2 edit
> ...
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
> atorRoot.java#2 edit
> ...
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
> intUtils.java#2 edit
> ...
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
> ource.java#2 edit
> ...
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
> ader.java#2 edit
> ...
> //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
> anceTest.java#2 edit
>
> Differences ...
>
> ====
> //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.j
> ava#2 (ktext) ====
>
> 2c2
> < // $Id:
> //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#1 $
> ---
> > // $Id:
> //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#2 $
> 27c27
> < * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#1 $
> ---
> > * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#2 $
> 227a228,232
> > * Returns an array of the non-All members which make
> up the current context
> > */
> > Member[] getNonAllMembers();
> >
> > /**
>
> ====
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
> gationManager.java#2 (ktext) ====
>
> 2c2
> < // $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
> gationManager.java#1 $
> ---
> > // $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
> gationManager.java#2 $
> 32c32
> < * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
> gationManager.java#1 $
> ---
> > * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
> gationManager.java#2 $
> 98c98
> < final Member[] currentMembers = evaluator.getMembers();
> ---
> > final Member[] currentMembers =
> evaluator.getNonAllMembers();
>
> ====
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
> ator.java#2 (ktext) ====
>
> 2c2
> < // $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
> ator.java#1 $
> ---
> > // $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
> ator.java#2 $
> 45c45
> < * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
> ator.java#1 $
> ---
> > * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
> ator.java#2 $
> 55a56
> > private TreeMap<Integer, RolapMember> nonAllMemberMap;
> 112a114,115
> > nonAllMemberMap =
> > (TreeMap<Integer, RolapMember>)
> parent.nonAllMemberMap.clone();
> 140a144,145
> > nonAllMemberMap =
> > (TreeMap<Integer, RolapMember>)
> root.nonAllDefaultMembers.clone();
> 223a229,233
> > public final Member[] getNonAllMembers() {
> > final Collection<RolapMember> members =
> nonAllMemberMap.values();
> > return members.toArray(new Member[members.size()]);
> > }
> >
> 383a394,398
> > if (m.isAll()) {
> >
> nonAllMemberMap.remove(m.getHierarchy().getOrdinalInCube());
> > } else {
> >
> nonAllMemberMap.put(m.getHierarchy().getOrdinalInCube(), m);
> > }
> 600,602c615,617
> < for (int i = 0; i < currentMembers.length; i++) {
> < final Member member = currentMembers[i];
> <
> ---
> > int i = -1;
> > for (RolapMember member : nonAllMemberMap.values()) {
> > i++;
>
> ====
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
> atorRoot.java#2 (ktext) ====
>
> 2c2
> < // $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
> atorRoot.java#1 $
> ---
> > // $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
> atorRoot.java#2 $
> 28c28
> < * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
> atorRoot.java#1 $
> ---
> > * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
> atorRoot.java#2 $
> 47a48
> > final TreeMap<Integer, RolapMember> nonAllDefaultMembers;
> 66a68
> > nonAllDefaultMembers = new TreeMap<Integer, RolapMember>();
> 91a94,97
> > if (!defaultMember.isAll()) {
> > nonAllDefaultMembers.put(
> > hierarchy.getOrdinalInCube(), defaultMember);
> > }
>
> ====
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
> intUtils.java#2 (ktext) ====
>
> 2c2
> < // $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
> intUtils.java#1 $
> ---
> > // $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
> intUtils.java#2 $
> 32c32
> < * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
> intUtils.java#1 $
> ---
> > * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
> intUtils.java#2 $
> 57c57
> < Member[] members = evaluator.getMembers();
> ---
> > Member[] members = evaluator.getNonAllMembers();
>
> ====
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
> ource.java#2 (ktext) ====
>
> 2c2
> < // $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
> ource.java#1 $
> ---
> > // $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
> ource.java#2 $
> 39c39
> < * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
> ource.java#1 $
> ---
> > * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
> ource.java#2 $
> 688c688
> < final Member[] members = evaluator.getMembers();
> ---
> > final Member[] members = evaluator.getNonAllMembers();
>
> ====
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
> ader.java#2 (ktext) ====
>
> 65c65
> < * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
> ader.java#1 $
> ---
> > * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
> ader.java#2 $
> 93c93
> < * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
> ader.java#1 $
> ---
> > * @version $Id:
> //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
> ader.java#2 $
> 1132c1132
> < final Member[] members = evaluator.getMembers();
> ---
> > final Member[] members = evaluator.getNonAllMembers();
>
> ====
> //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
> anceTest.java#2 (ktext) ====
>
> 2c2
> < // $Id:
> //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
> anceTest.java#1 $
> ---
> > // $Id:
> //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
> anceTest.java#2 $
> 22c22
> < * @version $Id:
> //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
> anceTest.java#1 $
> ---
> > * @version $Id:
> //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
> anceTest.java#2 $
> 214a215,354
> > /***
> > * Tests performance of a larger schema with a large
> number of result cells
> > * Runs in 12.2 seconds when RolapEvaluator.getProperty
> uses currentMemmber
> > * Runs in 7.5 seconds when RolapEvaluator.getProperty
> uses nonAllMemberMap
> > * The performance boost gets more significant as the
> schema size grows
> > */
> > public void testBigResultsWithBigSchemaPerforms() {
> > TestContext testContext =
> TestContext.createSubstitutingCube(
> > "Sales",
> > "<Dimension name=\"Gender2\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender3\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender4\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender5\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender6\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender7\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender8\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender9\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender10\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender11\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender12\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender13\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender14\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender15\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender16\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender17\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender18\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender19\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>"
> > + "<Dimension name=\"Gender20\"
> foreignKey=\"customer_id\">\n"
> > + "<Hierarchy hasAll=\"true\"
> allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
> > + " <Table name=\"customer\"/>\n"
> > + " <Level name=\"Gender\"
> column=\"gender\" uniqueMembers=\"true\"/>\n"
> > + " </Hierarchy>"
> > + "</Dimension>",
> > null);
> > String mdx = "with "
> > + " member [Measures].[one] as '1'"
> > + " member [Measures].[two] as '2'"
> > + " member [Measures].[three] as '3'"
> > + " member [Measures].[four] as '4'"
> > + " member [Measures].[five] as '5'"
> > + " select "
> > +
> "{[Measures].[one],[Measures].[two],[Measures].[three],[Measur
> es].[four],[Measures].[five]}"
> > + " on 0, "
> > +
> "Crossjoin([Customers].[name].members,[Store].[Store Name].members)"
> > + " on 1 from sales";
> > long start = System.currentTimeMillis();
> > testContext.executeQuery(mdx);
> > printDuration("getProperty taking a long time", start);
> > }
> >
>
More information about the Mondrian
mailing list