[Mondrian] Refactor to change Member by List<Member>
wgorman at pentaho.org
Tue Nov 13 15:06:40 EST 2007
After doing a quick search on SqlMemberSource.getMembers(), Luis's
proposed changes shouldn't conflict with the work I'm doing.
Digging into it, I don't see where in Mondrian that method is called for
the SqlMemberSource class. I see that it is called in RolapCube to get
the measures, but never for any other purpose.
While I agree that this should be moved to a list, I don't know if this
code is causing performance issues. I suggest looking at the
SqlTupleReader.Target class for where the issues you are seeing may be
Hope that helps!
On Tue, 2007-11-13 at 11:22 -0800, Julian Hyde wrote:
> > Luis Canals wrote:
> > After taking a look at SqlMemberSource, we have discovered that all
> > elementes from a ResultSet from a JDBC query are loading into memory
> > and stored into an array of Member objects....
> > As we are wanting to deal with very very very big dimensions, the
> > first thing we need is not to overload the memory, so we are thinking
> > of changing the type returned by SqlMemberSource.getMembers from a
> > Member to a List<Member> to be able to change it, in future, to a
> > more intelligent List supported by an Iterator which takes elements
> > from ResultSet non demand.
> > Yes, we know this is going to move us to change a lot elements into
> > code, but using a List or an Iterator instead of an array of Members
> > is, in almost all places, a better approach.
> > We would like to know your oppinions and suggestions.
> Sounds like a good idea. Further, I have found that passing in a list is
> often better than returning a list, because this way the caller gets to have
> a say how the list is represented; and can easily make multiple calls and
> append to an existing list.
> That said, I think Will is making some sweeping changes in that area which
> he has not yet checked in. Will, please comment whether Luis's proposed
> change is likely to conflict with your pending change. If so, I will ask
> Luis to make the change only after you have checked in.
> It is possible that this change affects public APIs. Please post a summary
> of any changes to public APIs (or which people might conceivably have
> implemented/overridden) after you have made your change. I will add this to
> the release notes.
> When you have completd this API change and come to use novel implementations
> of lists, note that Richard Emberson has done some work to (a) use
> memory-sensitive lists, and (b) use iterators. Please refer to his work and
> use the same data structures if possible.
More information about the Mondrian