History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: UP-1040
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Critical Critical
Assignee: Eric Dalquist
Reporter: Keith Zantow
Votes: 4
Watchers: 4
Operations

If you were logged in you would be able to see more operations.
uPortal

PortletPreferences between multiple instances of a Portlet are unreliable

Created: 03/May/05 12:42 PM   Updated: 03/Jul/07 04:11 PM
Component/s: Portlet Container
Affects Version/s: 2.4.2, 2.5.0 M1, 2.5.1 RC1, 2.5.0 RC1, 2.5.0 RC3, 2.5.0 RC2, 2.5.0 GA, 2.5.1 RC2, 2.5.2 RC1, 2.4.3, 2.5.1 RC3, 2.5.1 GA, 2.5.3 RC1, 2.5.2 GA, 2.5.3 RC2, 2.5.3 RC3, 2.5.3 GA
Fix Version/s: 2.5.4, 2.6.0 RC1

Original Estimate: Unknown Remaining Estimate: Unknown Time Spent: Unknown
File Attachments: 1. Zip Archive 1040-FIX-save-prefs.zip (29 kb)
2. Zip Archive ModifiedSource.zip (23 kb)
3. Zip Archive source-and-diffs-from-up251.zip (16 kb)

Environment: Fedora/Debian Linux, Tomcat 5.0.28, MySQL 4.1.7, HSQLDB 1.7.2


 Description  « Hide
When publishing a portlet multiple times, specifying different parameters for each instance, the parameters are not passed correctly. It is not consistent how the parameters are passed. It appears that the functionality is mostly there to do just that, as the database appears to get populated correctly in the UP_PORTLET tables based on the portlet's channel id. When it comes to the parameters being passed to the portlet at render time, however, it is inconsistent what parameters they receive. Sometimes they get the correct parameters, other times they get the same parameters from any one of the channel instances.

I have included a portlet that displays the behavior I have described on both of our uPortal instances. It has an install script that should automatically compile and publish 2 channels (Preferences Test 1 & 2) with different settings, they just need to be added to your layout. It just needs the environment variables: UPORTAL_HOME=<up_src_dir> and TOMCAT_HOME=<tomcat_dir>.

http://filebox.vt.edu/users/kzantow/preferences-test.tar.gz


 All   Comments   Work Log   Change History   FishEye      Sort Order:
Keith Zantow [09/May/05 05:22 PM]
I tracked the problem down to primarily the CPortletAdapter class. Unfortunately, there is no easy solution for the problem. Here is what's happening:

Each Portlet instance will call CPortletAdapter.initPortletWindow(), which in turn will get the PortletDefinitionImpl for a given portletDefinitionId. Since the two portlets have the same portletDefinitionId (say "foo.Bar"), they get the same object returned and proceed to call the PortletDefinitionImpl.setChannelDefinition() method, associating the single Portlet object with the channel instance data.

Each time that a Portlet instance does this, it then calls PortletDefinitionImpl.loadPreferences(), which appends to or overwrites a previous portlet's definition preferences.

There are a number of additional problems with this, causing it to be difficult to fix. It appears each Portlet - not each Portlet instance - has its own CPortletAdapter instance, so when the portal calls CPortletAdapter.isCacheValid(), it is also shared among each Portlet. Also, the synchronization should have some work done.

The "fix" I tested disables caching, as it appears this would be a significant amount of work to fix. The workaround that I found is as follows:

in PortletDefinitionImpl.java, on line 215 (in loadPreferences()) add:
preferences = new PreferenceSetImpl();

in CPortletAdapter.java, line 532 (after try { in getMarkup()) add:

PortletWindowImpl wnd = (PortletWindowImpl)cd.getPortletWindow();
PortletEntityImpl ent = (PortletEntityImpl)wnd.getPortletEntity();
PortletDefinitionImpl def = (PortletDefinitionImpl)ent.getPortletDefinition();
ChannelDefinition channelDefinition = ChannelRegistryStoreFactory.getChannelRegistryStoreImpl().getChannelDefinition(Integer.parseInt(sd.getChannelPublishId()));
def.setChannelDefinition(channelDefinition);
def.loadPreferences();

That will load the preferences each time a channel is rendered.

Also, disable caching in CPortletAdapter.java, line 638 (in isCacheValid()), make it return true;

I also synchronized both methods getMarkup() and initPortletWindow() with an inner synchronized(this){} block instead of function-level synchronization... I had to remove some inner synchronized() blocks to avoid deadlock. I'm not sure how the function calls are ordered, but that could pose a problem. It seems to work fine for me, however.


Keith Zantow [25/May/05 08:50 AM]
Not sure what I was thinking...

To disable caching, in CPortletAdapter.java, line 638 (in isCacheValid()), make it return false;

FALSE, not true...


David Grimwood [18/Nov/05 05:51 PM]
This also affects uPortal 2.5x.

Andrew Petro [20/Nov/05 11:08 PM]
Updated affects-version per Dave's comment to reflect afffecting existing 2.x releases. Placed on 2.5.x roadmap.

Keith Zantow [06/Dec/05 01:12 PM]
This looks like it may be a duplicate of: UP-863

Eric Dalquist [11/Jan/06 03:43 PM]
Description of UP-1040

Keith described the problem well, I'm going to see if I can add a bit more detail to the problem and describe what would need to be done to fix it.

The source of the problem is the portlet domain object architecture of uPortal 2. To start the JSR-168 portlet specification & Pluto, the reference JSR-168 container, define the following domain object levels for portlets

PortletDeployment
-Generally this equates to each <portlet> tag in the portlet.xml.
-uPortal 2 does not have a PortletDeployment object.

PortletDefinition
-Generally this equates to publishing a PortletDeployment in the portal.
-uPortal 2 groups this both the deployment information and the publishing information into this object.

PortletEntity
-Generally this equates to a user subscribing to a PortletDefinition.
-uPortal 2 follows the general practice.

PortletWindow
-Generally this equates to a user placing a PortletEntity on their layout.
-uPortal 2 follows the general practice.

From those descriptions we can base the problem description. The way preferences are advertised to work in uPortal 2 has three layers. The base layer is defined in the portlet.xml, the next layer is defined during publishing in ChannelManager, the final layer is defined by the portlet code while the user is interacting with it.

This layering is supposed to allow increasing levels of customization for a portlet. Example: A weather portlet could have a default location ZIP code in the portlet.xml. The administrator then publishes the portlet multiple times specifying different ZIP codes for each published portlet. The end user can override the published ZIP code to get weather they are interested in. None of these saved ZIP codes should interfere.

With the current uPortal 2 codebase there is only one PortletDefinition per PortletDeployment instead of one PortletDefinition per published instance of the PortletDeployment. This one PortletDefinition object is where the one definition level portlet preferences object is stored.

At the beginning of rendering each portlet loads its definition and entity preferences from the backing store. If there are multiple portlets based on the same deployment on a single tab they are all using the same PortletDefinition object. Each rendering portlet would load the definition preferences associated with its channel ID into the one PortletDefinition object. Now there is one portlet preferences object which contains some non-deterministic layering of preferences from all of the portlets based on that portlet deployment on that tab.

Now that the shared PortletDefinition has been identified as the problem a fix needs to be determined.

The correct approach to resolving this issue would be to rename the uPortal 2 PortletDefinition object to PortletDeployment. The majority of the information it provides is actually from the portlet.xml. A new PortletDefinition should be created and keyed on the channel definition ID. This object would provide the preferences for the definition level.

A quick work-around that does not involve modifying uPortal code is to create a portlet deployment entry for each portlet you would like to publish. Instead of publishing 'WeatherPortletApp.3DayForcastPortlet' multiple times the portlet XML would define multiple, uniquely named portlets '3DayForcastPortlet1', '3DayForcastPortlet2', ... and each would be published just once.


Andrew Petro [02/Oct/06 01:59 AM]
Raising priority of this bug. This bug is very very annoying, and is blocking interesting uses of end-user-configurable portlets, such as RSS reader portlets, weather status portlets, dashboardy widgets, and the like.

Tim Carroll [06/Nov/06 04:00 PM]
i was trying to determine a temporary work-around to this issue a few weeks ago. i did not have much time to concentrate on it, but i determined the main problem to be the preservation of the sequence of calls in the CPortletAdapter rendering pipeline. Nameley: initPortletWindow() and getMarkup()

i thought this could be fixed by making renderCharacters a synchronized method; however, initPortletWindow() is also called from getRuntimeData(), and this seems to be the method that always makes the call.

the temp fix i used, is to synchronize the renderCharacters method and force initPortletWindow to be called everytime from within renderCharacters. in other words, i added the "synchronized" keyword to the renderCharacters method and commented the "if" logic that surrounds the initPortletWindow call.

can anyone comment on the overall impact of this as a temp solution? could this cause serious performance problems, or have any other adverse affects?

note: there may be an additional need to force initPortletWindow to be called everytime there is a multi-published portlet on the user's layout? further testing should be done to determine this.


Christopher J. Holdorph [29/Nov/06 02:04 PM]
Yes synchronizing that much will have serious performance issues, and possibly could cause deadlock in the render phase. UP-1424 had to remove the synchronize keyword from getMarkup to prevent this deadlock problem from occurring under high loads. I would try porting the NON IMultithreaded version from HEAD to see if that one works any better.

Tim Carroll [29/Nov/06 02:48 PM]
After further investigation, coding, and testing, I can confirm Eric's analysis. However, I believe that there is a slightly less intrusive fix that would serve as a viable solution for the up2 code set.

The current architecture and database structure can adequately support the separation of portlet definition, publication, and subscription level preferences; however, the current code does not adequately manage the loading of preferences to allow preservation of this separation during the users session.

Today, preferences are loaded in the following three phases:
1. The PortletDefinition object is loaded with preference values that were specified in the portlet.xml file at deploy time.
2. The PortletDefinition object is then loaded with preference values that were specified in the chanpub.xml file at publish time. These preference relationships are stored in the PORTLET_DEFINITION_PREFS and PORTLET_PREF_VALUES tables in the database. At runtime, they override any preferences of the same name that were previously loaded from the portlet.xml file.
3. The PortletEntity object is loaded with preference values that were specified by the user during or after subscription time. These preference relationships are stored in the PORLET_ENTITY_PREFS and PORTLET_PREF_VALUES tables in the database.

The PortletDefinition object is keyed on portletDefinitionId, which is unique to a portlet deployment descriptor specified in the portlet.xml. However, the portlet publication is keyed on chan_id, which is unique to a portlet publication descriptor specified in the chanpub.xml. There is a one (portletDefinitionId) to many (chan_id) relationship between these two keys. Therefore, in phase two of the preferences load, the publication preferences for a given chan_id are overwriting any previous publication preferences sharing that specific portletDefinitionId.

At render time, PortletEntity preferences override any preferences of the same name that were previously loaded from the PortletDefinition object. Because these are stored in two separate objects at runtime, no overwrites occur. This explains why user subscription preferences have never been an issue, while publication preferences have been a problem.

To correct this, we can treat the publication preferences (stored in PORTLET_DEFINITION_PREFS) as default values in the PortletEntity preferences (stored in PORTLET_ENTITY_PREFS), rather than overrides to the values in PortletDefinition preferences.

In the future, preferences would be loaded in the following three phases:
1. As before, the PortletDefinition object is loaded with preference values that were specified in the portlet.xml file at deploy time.
2. The PortletEntity object is loaded with preference values that were specified in the chanpub.xml file at publish time. These preference relationships would still be stored in the PORTLET_DEFINITION_PREFS and PORTLET_PREF_VALUES tables in the database.
3. The PortletEntity object is loaded with preference values that were specified by the user during or after subscription time. These preference relationships are stored in the PORLET_ENTITY_PREFS and PORTLET_PREF_VALUES tables in the database. At runtime, they override any preferences of the same name that were previously loaded from the PORTLET_DEFINITION_PREFS database table.

At render time, PortletEntity preferences would still override any preferences of the same name that were previously loaded from the PortletDefinition preferences. However, now the publication preferences would be applied to the unique instance of the subscription rather than the shared instance of the definition.

The only foreseeable drawback to this solution highlights the key difference between this solution and Eric's analysis:

The publication preferences would be loaded for each subscription of the portlet prior to applying user subscription preferences. This would not cause performance degradation from today's baseline, because the same preference loads occur currently (the preferences are just being loaded into different objects than before, to allow more predictable results). However, Eric's recommended solution would create an object that allowed the true separation of publication from subscription, and this may provide a slight performance enhancement over either method described herein.

However, the benefit is an easily implemented fix to the bug with no anticipated impact to function or performance. A coded solution is attached as ModifiedSource.zip


Tim Carroll [29/Nov/06 02:48 PM]
per comment on 11/29/2006

Jason Shao [05/Dec/06 09:23 AM]
Tim, this approach appears very promising to me. From a conceptual angle, treating the published parameters as defaults seems very appropriate, given the Portlet preferences model.

I'm at the JA-SIG conference right now, but will try to take a closer look at your patch when I get back.


Eric Dalquist [05/Dec/06 09:41 AM]
I'm trying to evaluate the attached code but there is no mention of what version of uPortal the fix should be applied to and much of the spacing and line endings in the files have changes from what is in CVS. If possible please create a unified diff from the base version of uPortal you are working with and attach the diff with a comment about what version of uPortal the diff is against.

Eric Dalquist [05/Dec/06 10:22 AM]
One issue with this patch is what happens when a portlet programmatically changes the PortletPreferences. The publish time preference values that are loaded into the entity preferences object would be persisted in the entity preferences table. Once this happens changing any existing preferences in the channel manage would have no affect on the portlet.

The entity level preferences will need to be physically layered instead of just loading the publish preferences and then the entity preferences into the same object. The persistence layer may also need to be updated to deal with this layered preferences system.

Also a quick guide for submitting patches is available: http://www.ja-sig.org/wiki/display/UPC/Contributing+Patches


Jason Shao [14/Dec/06 01:45 PM]
Would some kind of mask for READ ONLY preferences be the answer? Something that prohibited user values for preferences the user can't modify from being persisted?

My thought being – if a user CAN change a preference, once they've been to the preferences screen I don't want to change the values without them initiating some action, seems that it breaks user expectations. And most instances where an admin wants to update preferences are preferences that I don't want the user to change, or am just providing a suggested default in which case picking up updates is perhaps not so important.


Eric Dalquist [14/Dec/06 01:54 PM]
A READ-ONLY marker on a preference won't help at all. The preference will still be copied into the entity preferences table marked as read-only. Even if the logic was changed to not persisted read-only preferences as entity preferences this patch as is would break the current state of functionality any existing portlet that had definition preferences and entity preferences. That option could be discussed for a MINOR release but not a PATCHES release.

The real solution, which should be possible with some work, is to re-work the entity model to store the preferences in a layered manner in memory. The definition preferences would get loaded into the base layer of the entity and then entity preferences would get loaded into the layer on top of that. Changes to preferences made by the portlet would only affect the top layer of preferences which would be persisted on a .store() call as the entity preferences.


Tim Carroll [12/Jan/07 11:40 AM]
attached is an updated set of fix files that accomodates the preference storage issues raised. the full code and diffs (from 2.5.1) are included.

this could use a second or third set of eyes and some additional testing.


Tim Carroll [08/Feb/07 01:20 PM]
the last attachment is not a viable fix.

i have another recommendation in the works... i will post it for review after a little more testing...


Tim Carroll [16/Mar/07 03:53 PM]
i have tested this fix, and i believe it to be a viable solution to this issue.

please test and provide feedback or recommended improvements.


Tim Carroll [16/Mar/07 03:56 PM]
the previous comment refers to the files in the source-and-diffs-from-up251.zip

i can't quite get used to this bug tracking tool. i don't understand why a comment made when saving an attachment, shows no association to the attachment.


Eric Dalquist [24/Apr/07 01:49 PM]
I'm going to review the latest patch this week.

Eric Dalquist [26/Apr/07 01:20 PM]
After just a code review the latest patch looks good. I'll be doing functional testing of the patch when I get back to Wisconsin next week.

Eric Dalquist [08/May/07 05:03 PM]
This looks like a viable patch. I've done testing with both definition and entity preferences on the 2.5-patches branch. I'll be doing testing on the trunk tomorrow and hopefully get this in for 2.6

Eric Dalquist [09/May/07 11:04 AM]
Resolved by a slightly modified version of Tim Carrol's latest patch.
2.5-patches change set: http://developer.ja-sig.org/source/changelog/jasigsvn/?cs=41891
trunk change set: http://developer.ja-sig.org/source/changelog/jasigsvn/?cs=41892

Jason Shao [09/May/07 11:12 AM]

Sweet!

Jason

Jason Shao
Application Developer
Office of Instructional & Research Technology
Rutgers University
v. 732-445-8726 | f. 732-445-5539 | jayshao@rutgers.edu