[Mondrian] RolapSchema #122

Matt Campbell mkambol at gmail.com
Wed Feb 18 14:32:47 EST 2009


I took silence for assent and have checked in 12382, which clears the VFS
cache for the requested file when this inconsistency is encountered.  I also
did some minor cleanup to eliminate some of the duplicate code.  Let me know
if that change looks reasonable to you.

On Wed, Feb 18, 2009 at 10:14 AM, Matt Campbell <mkambol at gmail.com> wrote:

> It does certainly seem to be a vfs bug.  As a workaround, I added the
> following block right after the .resolveFile().  This validates that the url
> passed in matches the url of the file retrieved, and if not clears the file
> from the cache and reloads.
>  if ( !file.getName().getURI().equals(url)) {
>             fsManager.getFilesCache().removeFile(file.getFileSystem(),
>  file.getName());
>             file = fsManager.resolveFile(userDir, url);
>         }
>
> Any objection to me checking in this change?  If we find a better
> workaround to the vfs bug we can always drop this check.
>
>
> On Tue, Feb 17, 2009 at 6:20 PM, Sherman Wood <swood at jaspersoft.com>wrote:
>
>>  I was just going to refactor RolapSchema.load() to use
>> Util.readVirtualFile() because of that identical code.
>>
>>
>>
>> The default configuration of Apache VFS we are using should be getting new
>> content each time a URI is "resolved" (
>> http://commons.apache.org/vfs/apidocs/org/apache/commons/vfs/CacheStrategy.html#ON_RESOLVE), which happens each time .readVirtualFile is called.
>>
>>
>>
>> I'll do some further tests, taking into account Will's comments about
>> possible VFS HTTP bugs.
>>
>>
>>
>>
>>
>> Sherman
>>
>>
>>
>> *From:* mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
>> *On Behalf Of *Matt Campbell
>> *Sent:* Wednesday, February 18, 2009 6:14 AM
>> *To:* Mondrian developer mailing list
>> *Subject:* Re: [Mondrian] RolapSchema #122
>>
>>
>>
>> I just noticed that RolapSchema.load() uses identical code to
>> Util.readVirtualFile().  It seems likely that it would have the same problem
>> with cached files.
>>
>> On Tue, Feb 17, 2009 at 12:18 PM, Matt Campbell <mkambol at gmail.com>
>> wrote:
>>
>> It looks like the parameters in the URL are ignored on the second call to
>> readVirtualFile().  The unit test below shows the issue.  The URI of the
>> second file returned by .readVirtualFile ends up being identical to uri1.
>>
>>
>>
>> Could you revert your change until this is fixed?
>>
>>
>>
>>  public void testReadVirtualFile() {
>>
>>         try {
>>
>>             String uri1 = "http://localhost:8088/?URI1";
>>
>>             String uri2 = "http://localhost:8088/?URI2";
>>
>>             FileObject file1 = Util.readVirtualFile(uri1);
>>
>>             FileObject file2 = Util.readVirtualFile(uri2);
>>
>>
>>
>>             assertEquals(file1.getName().getURI(), uri1);
>>
>>             assertEquals(file2.getName().getURI(), uri2);
>>
>>         } catch(Exception e) {
>>
>>              fail("Threw an exception while reading virtual file");
>>
>>         }
>>
>>     }
>>
>>
>>
>> On Mon, Feb 16, 2009 at 4:36 PM, Sherman Wood <swood at jaspersoft.com>
>> wrote:
>>
>>
>>
>> Yep – that looks weird and bad. I'll look into it too.
>>
>>
>>
>> Sherman
>>
>>
>>
>> *From:* mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
>> *On Behalf Of *Matt Campbell
>> *Sent:* Tuesday, February 17, 2009 4:01 AM
>> *To:* Mondrian developer mailing list
>> *Subject:* [Mondrian] RolapSchema #122
>>
>>
>>
>> Sherman,
>>
>> It looks like a bug was introduced in the code which retrieves schemas
>> from the Pool.  In Pool.get() there is code which first loads the schema
>> from a given URL, then computes its checksum.  I've noticed that the new
>> Util.readVirtualFile() method will sometimes return the content of the
>> *previous* schema loaded.
>>
>>
>>
>> In the current code there is actually an extra read of the catalog--see
>> the bolded section below.  It's leftover from the previous approach for
>> loading the schema.  This should probably be cleaned up, but it helped me
>> find the issue.  When I've traced through this code I've seen that the
>> catalogStr returned by the old code (Util.readURL) is sometimes different
>> from the schema returned by Util.readVirtualFile.
>>
>>
>>
>> I'm going to try to come up with a clean reproducible test case tomorrow,
>> but I thought I'd pass on what I've seen in case you see something obviously
>> wrong.
>>
>>
>>
>>                 try {
>>
>>                     if (catalogStr == null) {
>>
>>                        * catalogStr = Util.readURL(catalogUrl);*
>>
>>                         // Use VFS to get the content
>>
>>
>>
>>                         FileObject file =
>> Util.readVirtualFile(catalogUrl);
>>
>>
>>
>>                         StringBuilder buf = new StringBuilder(1000);
>>
>>                         FileContent fileContent = file.getContent();
>>
>>                         InputStream in = fileContent.getInputStream();
>>
>>                         int n;
>>
>>                         while ((n = in.read()) != -1) {
>>
>>                             buf.append((char) n);
>>
>>                         }
>>
>>                      *   catalogStr = buf.toString();*
>>
>>                     }
>>
>>                     md5Bytes = encodeMD5(catalogStr);
>>
>>                 } catch (Exception ex) {
>>
>>                     // Note, can not throw an Exception from this method
>>
>>                     // but just to show that all is not well in Mudville
>>
>>                     // we print stack trace (for now - better to change
>>
>>                     // method signature and throw).
>>
>>                     ex.printStackTrace();
>>
>>                 }
>>
>>
>>
>> -Matt
>>
>>
>>
>> _______________________________________________
>> Mondrian mailing list
>> Mondrian at pentaho.org
>> http://lists.pentaho.org/mailman/listinfo/mondrian
>>
>>
>>
>>
>>
>> _______________________________________________
>> Mondrian mailing list
>> Mondrian at pentaho.org
>> http://lists.pentaho.org/mailman/listinfo/mondrian
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20090218/072b3138/attachment.html 


More information about the Mondrian mailing list