[Mondrian] Failure to get table cardinality for large fact table

Kaushik Chandrashekar kaushik at zedo.com
Tue Apr 18 05:55:36 EDT 2017


Hi Julian,

I tried logging a JIRA issue, but had errors doing so.
_______________________________

Details are:

*Environment:*
Mondrian version:4.6.0.0-25
Olap4j version: 1.2.0

*Steps to reproduce:*
- Fact table should contain rows greater than Integer max value
(2,147,483,647)
- Fact table to be declared in PhysicalSchema
<PhysicalSchema>
    <Table name="very_large_fact_table"/>
</PhysicalSchema>
- Establishing a connection will cause failure at SqlStatisticsProvider

*Description:*
The interface mondrian.spi.StatisticsProvider declares return types of
methods:
- getTableCardinality
- getQueryCardinality
- getColumnsCardinality
as int. This sets an upper bound on values to 2,147,483,647

The implementing class mondrian.spi.impl.SqlStatisticsProvider uses a
java.sql.ResultSet getInt() method to retrieve these values. This would
break when the output of any of these queries exceeds the above value.
_______________________________

While I'm not completely familiar with the mondrian codebase and hence
might not be able to produce a fix PR, possible fix might be to change
interface mondrian.spi.StatisticsProvider, method getTableCardinality
return type to long and corresponding impl to use getLong() of resultSet. I
am not aware of the other components affected by change to this interface.

Regards,
Kaushik



> Date: Thu, 13 Apr 2017 12:37:30 -0700
> From: Julian Hyde <julianhyde at gmail.com>
> Subject: Re: [Mondrian] Failure to get table cardinality for large
>         fact table
> To: Mondrian mailing list <mondrian at pentaho.org>
> Message-ID: <52621AC9-0053-43D0-A919-8D9833C88965 at gmail.com>
> Content-Type: text/plain; charset="us-ascii"
>
> Can you log a JIRA case? Also supply a PR if you have a fix.
>
> > On Apr 13, 2017, at 12:32 AM, Kaushik Chandrashekar <kaushik at zedo.com>
> wrote:
> >
> > Hi all,
> >
> > I am developing an application using mondrian 4 (4.6.0.0-25) with olap4j.
> > Out datasource has a few fact tables having row counts in the range of
> 19 billion rows.
> >
> > The function to get table cardinality uses int and not long. This puts a
> constraint on max row size of any table to be lesser than 2,147,483,647 (a
> little over 2 billion). Class/method ref: mondrian.spi.impl.
> SqlStatisticsProvider.getTableCardinality(SqlStatisticsProvider.java:52)
> >
> > I get the following failure as a consequence: com.mysql.jdbc.exceptions.jdbc4.MySQLDataException:
> '1.9001263577E10' in column '1' is outside valid range for the datatype
> INTEGER
> >
> > How do I go about this issue?
> >
> > Thanks
> > _______________________________________________
> > Mondrian mailing list
> > Mondrian at pentaho.org
> > http://lists.pentaho.org/mailman/listinfo/mondrian
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20170418/b0134888/attachment.html 


More information about the Mondrian mailing list