[Mondrian] RolapSchema #122
Matt Campbell
mkambol at gmail.com
Wed Feb 18 14:32:47 EST 2009
I took silence for assent and have checked in 12382, which clears the VFS
cache for the requested file when this inconsistency is encountered. I also
did some minor cleanup to eliminate some of the duplicate code. Let me know
if that change looks reasonable to you.
On Wed, Feb 18, 2009 at 10:14 AM, Matt Campbell <mkambol at gmail.com> wrote:
> It does certainly seem to be a vfs bug. As a workaround, I added the
> following block right after the .resolveFile(). This validates that the url
> passed in matches the url of the file retrieved, and if not clears the file
> from the cache and reloads.
> if ( !file.getName().getURI().equals(url)) {
> fsManager.getFilesCache().removeFile(file.getFileSystem(),
> file.getName());
> file = fsManager.resolveFile(userDir, url);
> }
>
> Any objection to me checking in this change? If we find a better
> workaround to the vfs bug we can always drop this check.
>
>
> On Tue, Feb 17, 2009 at 6:20 PM, Sherman Wood <swood at jaspersoft.com>wrote:
>
>> I was just going to refactor RolapSchema.load() to use
>> Util.readVirtualFile() because of that identical code.
>>
>>
>>
>> The default configuration of Apache VFS we are using should be getting new
>> content each time a URI is "resolved" (
>> http://commons.apache.org/vfs/apidocs/org/apache/commons/vfs/CacheStrategy.html#ON_RESOLVE), which happens each time .readVirtualFile is called.
>>
>>
>>
>> I'll do some further tests, taking into account Will's comments about
>> possible VFS HTTP bugs.
>>
>>
>>
>>
>>
>> Sherman
>>
>>
>>
>> *From:* mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
>> *On Behalf Of *Matt Campbell
>> *Sent:* Wednesday, February 18, 2009 6:14 AM
>> *To:* Mondrian developer mailing list
>> *Subject:* Re: [Mondrian] RolapSchema #122
>>
>>
>>
>> I just noticed that RolapSchema.load() uses identical code to
>> Util.readVirtualFile(). It seems likely that it would have the same problem
>> with cached files.
>>
>> On Tue, Feb 17, 2009 at 12:18 PM, Matt Campbell <mkambol at gmail.com>
>> wrote:
>>
>> It looks like the parameters in the URL are ignored on the second call to
>> readVirtualFile(). The unit test below shows the issue. The URI of the
>> second file returned by .readVirtualFile ends up being identical to uri1.
>>
>>
>>
>> Could you revert your change until this is fixed?
>>
>>
>>
>> public void testReadVirtualFile() {
>>
>> try {
>>
>> String uri1 = "http://localhost:8088/?URI1";
>>
>> String uri2 = "http://localhost:8088/?URI2";
>>
>> FileObject file1 = Util.readVirtualFile(uri1);
>>
>> FileObject file2 = Util.readVirtualFile(uri2);
>>
>>
>>
>> assertEquals(file1.getName().getURI(), uri1);
>>
>> assertEquals(file2.getName().getURI(), uri2);
>>
>> } catch(Exception e) {
>>
>> fail("Threw an exception while reading virtual file");
>>
>> }
>>
>> }
>>
>>
>>
>> On Mon, Feb 16, 2009 at 4:36 PM, Sherman Wood <swood at jaspersoft.com>
>> wrote:
>>
>>
>>
>> Yep – that looks weird and bad. I'll look into it too.
>>
>>
>>
>> Sherman
>>
>>
>>
>> *From:* mondrian-bounces at pentaho.org [mailto:mondrian-bounces at pentaho.org]
>> *On Behalf Of *Matt Campbell
>> *Sent:* Tuesday, February 17, 2009 4:01 AM
>> *To:* Mondrian developer mailing list
>> *Subject:* [Mondrian] RolapSchema #122
>>
>>
>>
>> Sherman,
>>
>> It looks like a bug was introduced in the code which retrieves schemas
>> from the Pool. In Pool.get() there is code which first loads the schema
>> from a given URL, then computes its checksum. I've noticed that the new
>> Util.readVirtualFile() method will sometimes return the content of the
>> *previous* schema loaded.
>>
>>
>>
>> In the current code there is actually an extra read of the catalog--see
>> the bolded section below. It's leftover from the previous approach for
>> loading the schema. This should probably be cleaned up, but it helped me
>> find the issue. When I've traced through this code I've seen that the
>> catalogStr returned by the old code (Util.readURL) is sometimes different
>> from the schema returned by Util.readVirtualFile.
>>
>>
>>
>> I'm going to try to come up with a clean reproducible test case tomorrow,
>> but I thought I'd pass on what I've seen in case you see something obviously
>> wrong.
>>
>>
>>
>> try {
>>
>> if (catalogStr == null) {
>>
>> * catalogStr = Util.readURL(catalogUrl);*
>>
>> // Use VFS to get the content
>>
>>
>>
>> FileObject file =
>> Util.readVirtualFile(catalogUrl);
>>
>>
>>
>> StringBuilder buf = new StringBuilder(1000);
>>
>> FileContent fileContent = file.getContent();
>>
>> InputStream in = fileContent.getInputStream();
>>
>> int n;
>>
>> while ((n = in.read()) != -1) {
>>
>> buf.append((char) n);
>>
>> }
>>
>> * catalogStr = buf.toString();*
>>
>> }
>>
>> md5Bytes = encodeMD5(catalogStr);
>>
>> } catch (Exception ex) {
>>
>> // Note, can not throw an Exception from this method
>>
>> // but just to show that all is not well in Mudville
>>
>> // we print stack trace (for now - better to change
>>
>> // method signature and throw).
>>
>> ex.printStackTrace();
>>
>> }
>>
>>
>>
>> -Matt
>>
>>
>>
>> _______________________________________________
>> Mondrian mailing list
>> Mondrian at pentaho.org
>> http://lists.pentaho.org/mailman/listinfo/mondrian
>>
>>
>>
>>
>>
>> _______________________________________________
>> Mondrian mailing list
>> Mondrian at pentaho.org
>> http://lists.pentaho.org/mailman/listinfo/mondrian
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pentaho.org/pipermail/mondrian/attachments/20090218/072b3138/attachment.html
More information about the Mondrian
mailing list