[Mondrian] Testing a Hive dialect

Calum Miller calum at millersoft.ltd.uk
Fri Feb 11 13:24:35 EST 2011

I have time next week to help make necessary Mondrian dialect changes. Happy to start on Monday if Jira is updated?


Sent from my iPhone

On 11 Feb 2011, at 17:51, "Julian Hyde" <jhyde at pentaho.com> wrote:

> If it is difficult to fix HIVE-1922, we could possibly workaround issue by changing Dialect.generateOrderItem. (Every dialect tends to have different rules for how to generate an ORDER BY clause -- order by column name, order by ordinal, order by expression, order by ordinal only when applied to a set operation such as union, etc. -- so there's no harm having yet another behavior.)
> Someone also mentioned that Hive only supports ANSI join syntax 'FROM t1 JOIN t2 ON t1.x = t2.y', whereas mondrian only generates 'FROM t1, t2 WHERE t1.x = t2.y'. Is this still an issue? We would need to fix either Hive or Mondrian's dialect. Mondrian's  dialect it probably easier.
> Since other people have expressed an interest in a Hive dialect, it would be useful if you check in what you have right now, even though there are many failures. Send me the files and I will check them in.
> Julian
> From: mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org] On Behalf Of Fu Hongwei
> Sent: Friday, February 11, 2011 3:53 AM
> To: 'Mondrian developer mailing list'
> Subject: Re: RE: [Mondrian] Testing a Hive dialect
> Hi Julian,
> Thanks for the prompt reply.
> The test is running on Hive 0.7.0, but a patch will be needed. I've started a jira on Hive and will submit it soon too.
> https://issues.apache.org/jira/browse/HIVE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12987548#action_12987548
> There are too many failures in the test suite.  I'm still working on it, but I will submit the DialectTest.java and HiveDialect.java, so people interested can work on it together.
> I think the point is first to make it running then speed it up.
> 2011-02-11
> Fu Hongwei
> 发件人: Julian Hyde
> 发送时间: 2011-02-11  00:19:26
> 收件人: 'Mondrian developer mailing list'
> 抄送:
> 主题: RE: [Mondrian] Testing a Hive dialect
> I am well aware of the compromises with Hive. There is certainly an impedance mismatch between hadoop and the real-time analysis, and that is reflected in query response time. It is still useful  to have a Hive dialect, because as you say, Hive is improving all the time. And Pentaho is thinking about ways to bridge the impedance mismatch.
> Can please you attach your code to the jira case as a patch? I will submit it.
> Also please attach the output of the test suite, and describe the version of Hive you are running    against. That will be a reference point for others who are working on Hive.
> Julian
> From: mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org] On Behalf Of Fu Hongwei
> Sent: Thursday, February 10, 2011 1:29 AM
> To: mondrian
> Subject: [Mondrian] Testing a Hive dialect 
> Hi,
> I'm new to the community. I don't know who to ask this for.
> I've written a Hive dialect for mondrian and been testing it.
> The DialectTest is passed, but it's still failing other tests.
> From what I can see it's still a long way to make an integration
> of Hive and Mondrian practical.
> 1. It's really slow. It takes 1 or 2 days to run through all the tests,
> a low end estimation. The latency is too big for most applications.
> It might take some major architectural change on the Hive side to solve
> this problem.
> 2. Hive ql is still at a rather immature stage, partly because it's not
> really intended to be a full featured relational database. There are some
>  bugs too, like the join behavior is incorrect at the current trunk.
> I know there is already a jira about it.
> http://jira.pentaho.com/browse/MONDRIAN-789 
> How can I submit the code? Thanks
> 2011-02-10
> Fu Hongwei
> _______________________________________________
> Mondrian mailing list
> Mondrian at pentaho.org
> http://lists.pentaho.org/mailman/listinfo/mondrian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20110211/94053cf5/attachment.html 

More information about the Mondrian mailing list