[Mondrian] Failure to get table cardinality for large fact table

Kaushik Chandrashekar kaushik at zedo.com
Tue Apr 18 05:55:36 EDT 2017

Hi Julian,

I tried logging a JIRA issue, but had errors doing so.

Details are:

Mondrian version:
Olap4j version: 1.2.0

*Steps to reproduce:*
- Fact table should contain rows greater than Integer max value
- Fact table to be declared in PhysicalSchema
    <Table name="very_large_fact_table"/>
- Establishing a connection will cause failure at SqlStatisticsProvider

The interface mondrian.spi.StatisticsProvider declares return types of
- getTableCardinality
- getQueryCardinality
- getColumnsCardinality
as int. This sets an upper bound on values to 2,147,483,647

The implementing class mondrian.spi.impl.SqlStatisticsProvider uses a
java.sql.ResultSet getInt() method to retrieve these values. This would
break when the output of any of these queries exceeds the above value.

While I'm not completely familiar with the mondrian codebase and hence
might not be able to produce a fix PR, possible fix might be to change
interface mondrian.spi.StatisticsProvider, method getTableCardinality
return type to long and corresponding impl to use getLong() of resultSet. I
am not aware of the other components affected by change to this interface.


> Date: Thu, 13 Apr 2017 12:37:30 -0700
> From: Julian Hyde <julianhyde at gmail.com>
> Subject: Re: [Mondrian] Failure to get table cardinality for large
>         fact table
> To: Mondrian mailing list <mondrian at pentaho.org>
> Message-ID: <52621AC9-0053-43D0-A919-8D9833C88965 at gmail.com>
> Content-Type: text/plain; charset="us-ascii"
> Can you log a JIRA case? Also supply a PR if you have a fix.
> > On Apr 13, 2017, at 12:32 AM, Kaushik Chandrashekar <kaushik at zedo.com>
> wrote:
> >
> > Hi all,
> >
> > I am developing an application using mondrian 4 ( with olap4j.
> > Out datasource has a few fact tables having row counts in the range of
> 19 billion rows.
> >
> > The function to get table cardinality uses int and not long. This puts a
> constraint on max row size of any table to be lesser than 2,147,483,647 (a
> little over 2 billion). Class/method ref: mondrian.spi.impl.
> SqlStatisticsProvider.getTableCardinality(SqlStatisticsProvider.java:52)
> >
> > I get the following failure as a consequence: com.mysql.jdbc.exceptions.jdbc4.MySQLDataException:
> '1.9001263577E10' in column '1' is outside valid range for the datatype
> >
> > How do I go about this issue?
> >
> > Thanks
> > _______________________________________________
> > Mondrian mailing list
> > Mondrian at pentaho.org
> > http://lists.pentaho.org/mailman/listinfo/mondrian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20170418/b0134888/attachment.html 

More information about the Mondrian mailing list