Academus uPortal Enhancements

As discussed at the Rutgers uPortal Developer Meeting, Unicon is looking to make available in opensource all of its modifications to uPortal itself that are in its product built on uPortal (Academus). Ideally all of these changes will be welcome improvements to uPortal and will be merged into opensource uPortal. Community review and feedback is welcome.

This page exists to summarize what these changes are. Additional details are available in Jira.

Motivation

Unicon has decided to put into opensource the uPortal framework changes in Academus. Ideally, and subject to developer community approval, these changes will also be merged into the opensource uPortal project so that by uPortal 2.6 most if not all of these changes are included in uPortal This will be good for Unicon (reduce merge cost) and will be good for the uPortal project (demonstrate life of project, build excitement, fix some annoying bugs, improve performance, add minor capabilities).

Marketing blurb

The latest 2.1 release of the Academus product is built on uPortal 2.5.3 with a number of modifications. In the interest of making Academus development more sustainable and in contributing to the quality and vibrancy of the uPortal project, Unicon is pleased to seek to contribute back to the core uPortal project all modifications to uPortal currently maintained for Academus. These modifications include enhancements to the flexibility and configurability of the uPortal framework, the richness of the channel environment, framework support for an additional RDBMS and for additional RDBMS features, user experience enhancements, significant performance improvements, and miscellaneous additional bugfixes.

The following outlines these changes at a high level:

• Provide channels the opportunity to be aware of the currently active serializer, enabling them to consistently serialize. This is necessary for self-serializing channels (JSR-168 portlets, character channels) to ensure their output format is appropriate in the larger context of the portal response.
• Guest users portlet caching becomes instance scoped rather than system scoped. This makes it possible for guest layouts to include more sophisticated portlets.
• Support Group API implementations that implement the appropriate interface but do not extend from GroupMemberImpl. (Currently this is not supported because the framework casts to an implementation instead of an interface.)
• Handle file uploading failures more cleanly. Introduce an UploadStatus object that can better communicate upload failures (as well as successes).
• Make the JNDI path to the uPortal datasource configurable. This adds support for placing it elsewhere than the read-only java:/comp/env space. The current hard-coding of this path presents difficulties for certain kinds of automated testing.
• Enhance the DbLoader to support SQL Server as well as BIGINT datatype.
• Enhance the ALM store to enable using foreign key constraints for db administrators that wish to do so. This entails modifying the order in which operations are performed. Support for lack of foreign key constraints can be retained if desired.
• Groups Manager channel enhancements
• Making user and channel searching case insensitive.

Additionally, a number of changes implement performance enhancements:

• Implement an attribute wrapper for HttpServletRequestWrapper, and make all current RequestWrappers extend from it. This solves concurrency problems as a result of sharing data via a Request Attribute (such as in Pluto). There are several areas of CPortletAdapter that can be optimized to avoid monitor bottlenecks.
• Remove a significant bottleneck in the user identity store by avoiding its current behavior of locking every user behind the getUserId method.
• Enable XSLT to use xsltc translets. Tests showed a 4x decrease in transformation time with using translets. Some stylesheets are not suitable for precompilation, a complication which can be addressed by compiling stylesheets at runtime and maintaining a preconfigured but dynamically updating blacklist of known-uncompilable stylesheets.
• Provide a new Stats Recorder implementation that logs in milliseconds instead of newing up a Date object for each log message. This enables the stats recorder to be efficiently used within a performance testing environment.
• Remove excessive object creation by carefully synchronizing around the principalCache, rather than cloning and replacing.
• Additional miscellaneous bug fixes...

The changes

Making Serializer available to Channels

Unicon-internal tracking number: TT 4903
uPortal JIRA UP-1503
This was necessary because channels can self-serialize and a client had a requirement to deliver xhtml. The serializer name is made available within the ChannelStaticData object.

• org/jasig/portal/ChannelStaticData
  • Added serializerName attribute and getter/setter
• org/jasig/portal/UserInstance
  • Added serializerName attribute
  • Passed the serializer name from the ThemeStylesheetDescription object to the ChannelManager object.
• org/jasig/portal/ChannelManager
  • Added serializerName attribute and getter/setter
  • Set's the serializerName in the ChannelStaticData object prior to passing it along to the channel.
• org/jasig/portal/MediaManager
  • Added configurable public and system IDs
  • Added setter for omitting doc type
• org/jasig/portal/channels/CError
  • tells MediaManager to omit document type
• org/jasig/portal/serialize/HTMLSerializer
  • Changed to use public and system IDs from OutputFormat

Implement a new GroupsManager permissions object that disallows the selection of top-level groups

Unicon-internal tracking number: TT 5184
uPortal JIRA UP-1504

• org/jasig/portal/channels/groupsmanager/permissions/GroupsManagerBlockEntityAndRootSelectPermissions
  • New file
• org/jasig/portal/channels/CChannelManager
  • Changed to use new permissions object

Unfocusing a PortletAdapter channel (any ICC channel) crashes uPortal

Unicon-internal tracking number TT 4959
uPortal JIRA UP-1561

• org/jasig/portal/ChannelManager
  • Added check for channelTarget within pending channels

The Group Manager servant would not exit if you chose to add members and then canceled your selection.

uPortal Jira UP-820.

• org/jasig/portal/channels/groupsmanager/commands/CancelSelection
  • removed groupParentId from session data

Fragment Manager - Duplicate information on info button when publishing fragment

Unicon-internal tracking number TT 5146

uPortal JIRA UP-1562

• org/jasig/portal/channels/groupsmanager/commands/ShowProperties
  • Added a check if properties element already exists

Groups Manager: A group without a description causes the XML to fail

Unicon-internal tracking number TT 5311
uPortal JIRA UP-1564

• org/jasig/portal/channels/groupsmanager/GroupsManagerXML
  • Check for empty strings prior to creating DOM TextNodes

Guest users share portlet cache

Unicon-internal TT 5152
uPortal JIRA UP-1565

• org/jasig/portal/channels/portlet/CPortletAdapter
  • Remove the special case handling for guest users; CPortletAdapter will now always use instance-level caching, rather than specifying system-level caching for guest users.

Concurrency fix in sharing data via request attributes

Unicon-internal tracking number TT 5266
uPortal JIRA UP-1566

Implement an attribute wrapper for HttpServletRequestWrapper, and make all current RequestWrappers extend from it. This solves concurrency problems as a result of sharing data via a Request Attribute (such as Pluto).

• org/jasig/portal/container/servlet/AttributeRequestWrapper
  • New object
• org/jasig/portal/channels/portlet/CPortletAdapter
  • streamlined synchronization
• org/jasig/portal/container/servlet/DummyParameterRequestWrapper
  • changed to extend from AttributeRequestWrapper
• org/jasig/portal/container/servlet/PortletParameterRequestWrapper
  • changed to extend from AttributeRequestWrapper
• org/jasig/portal/container/servlet/ServletRequestImpl
  • changed to extend from AttributeRequestWrapper

CPortletAdapter Performance enhancement

Unicon-internal tracking number TT 5265
uPortal JIRA UP-1567

• org/jasig/portal/channels/portlet/CPortletAdapter
  • Save on memory utilization in the renderCharacters() method of CPortletAdapter by using the PrintWriter parameter directly in getMarkup().

Custom Groups Implementation Support

Unicon-internal tracking number TT 4918
uPortal JIRA UP-1568

This is support for alternate implementations of the groups API.

• org/jasig/portal/groups/GroupMemberImpl
  • Re-implemented primGetAllContainingGroups to allow integration with group implementation that do not extend from it.

Making Groups Manager user search case insensitive

Unicon-internal tracking number TT 5038
JA-SIG uPortal JIRA UP-1505

• org/jasig/portal/groups/ldap/LDAPGroupStore
• org/jasig/portal/groups/local/searchers/RDBMChannelDefSearcher
• org/jasig/portal/groups/local/searchers/RDBMPersonSearcher
• org/jasig/portal/groups/RDBMEntityGroupStore
  • Changed to ignore case in searches

Don't allow a Fragment to be added to a Fragment.

Unicon-internal tracking number TT 4856
JA-SIG uPortal JIRA UP-1569
The chosen solution requires the 'isFragment' flag to be passed from the preferences xslt to the CContentSubscriber channel. CContentSubscriber will then allow or disallow the addition of fragments based on the value of that flag.

• org/jasig/portal/layout/alm/channels/CContentSubscriber
  • passing 'isFragment' information to stylesheets

Prohibit Publishing Fragments to Individual Users

Unicon-internal tracking number TT 5040
JA-SIG uPortal JIRA UP-1570

• org/jasig/portal/layout/alm/channels/FragmentManager
  • removed analyzeParameters method
  • added inner class ErrorMessage to be able to pass messages to callers if trying to add a fragment
• org/jasig/portal/layout/alm/channels/CFragmentManager
  • refactored analyzedParameters method to return an ErrorMessage object to communicate status of operation.

Passing upload status to channels

JA-SIG uPortal JIRA UP-1571
When a channel was uploading a particular file and that file size exceeded the maximum allowable size, it would cause the channel to fail. This change passes a status object to channels that they can query and display an error message if necessary. Academus uses its own implementations of PortalSessionManager and RequestParamWrapper. It looks like this can be removed because 2.5.2 has this functionality now, but it's not as flexible.

• org/jasig/portal/PortalSessionManager
  • implemented getter methods for fatalError, ALLOW_REPEATED_REQUESTS and randomGenerator to make it extendable.
• net/unicon/uportal/UniconPortalSessionManager
  • augmented from PortalSessionManager to utilize UniconRequestParamWrapper object
• net/unicon/uportal/UniconRequestParamWrapper
  • augmented from RequestParamWrapper to pass UploadStatus object to channels performing uploads
• net/unicon/uportal/UploadStatus
  • new file

Removed jndi DataSource base name hardcoding

Unicon-internal tracking number TT 5011
JA-SIG uPortal JIRA UP-1572
This was needed to be able to pass a data source from other than the read-only context java:/env/comp .. for testing purposes.

• org/jasig/portal/RDBMServices
  • made jndi base context configurable

Locking in RDBMUserIdentityStore

JA-SIG uPortal JIRA UP-1573
There needs to be an update to RDBMUserIdentityStore. When updating group memberships in addNewUser and updateUser, it needs to work on a lockable group to avoid invalid group object updates.

RDBMUserIdentityStore getPortalUID only locked per user

JA-SIG uPortal JIRA UP-1574

• org/jasig/portal/RDBMUserIdentityStore
  • getPortalUID only locked per user

UserInstance cross-user cache hits

Unicon-internal tracking number TT 4858
JA-SIG uPortal JIRA UP-1575

• org/jasig/portal/UserInstance
  • Set system caches as static objects and removed username from the cachekey to enable cross user cache hits

Enable XSLT to use xsltc translets.

Unicon-internal tracking number: TT 4858

JA-SIG JIRA UP-1507
Tests showed a 4x increase in transformation time with using translets.

• org/jasig/portal/utils/XSLT
  • Added the functionality to use a xsltc transformer handler if applicable. Stylesheets will be dynamically compiled if:
    • The stylesheet is not blacklisted
    • The stylesheet or any included or imported stylesheet has a newer timestamp than does the translet.
  • A stylesheet will be blacklisted and will default to the default transformer handler if:
    • The stylesheet is within a blacklist property.
    • The stylesheet fails compilation.
    • The stylesheet transformation fails.

Academus performance tweaks

Unicon-internal tracking number TT 5169

JA-SIG uPortal JIRA UP-1506

• org/jasig/portal/security/provider/RDBMPermissionImpl
  • Reimplement the select() method to utilize prepared statements, which vastly improves repeated query execution.

uPortal Stats Recorder using too much CPU Time

Unicon-internal tracking number TT 5298

JA-SIG uPortal JIRA UP-1508
During performance tests we heavily depend on the stats recorder and it was scewing results.

• org/jasig/portal/services/stats/MessageStatsRecorder
  • Replace 'new Date()' with 'System.currentTimeMillis()'.
  • Remove 'fixMsg()' method, which required an additional StringBuffer to be created.

IFrameChannel does not detect IE7

Unicon-internal tracking number TT 5805

This issue has already been resolved in opensource uPortal. Task here is to compare Academus solution and ensure opensource solution is providing maximal goodness.

IE7 users would see a "This browser does not support inline frames." message when IFrame Channels were rendered.
LOE - 1 hour

• org/jasig/portal/channels/CInlineFrame
  • Added "MSIE 7.0" in getStylesheetTitle()

DB Loader changes to support different platforms.

JA-SIG uPortal JIRA UP-1576

• org/jasig/portal/tools/dbloader/DataHandler
• org/jasig/portal/tools/dbloader/DbLoader
• org/jasig/portal/tools/dbloader/DbUnload
• org/jasig/portal/tools/dbloader/DbUtils
• org/jasig/portal/tools/dbloader/TableHandler
  • Added changes for creating scripts
  • Added support for BIGINT type
• org/jasig/portal/tools/dbloader/DataHandlerFactory
• org/jasig/portal/tools/dbloader/PostgresDbUnload
• org/jasig/portal/tools/dbloader/SqlServerDataHandler
  • new files

Miscellaneous changes

• org/jasig/portal/layout/alm/AggregatedUserLayoutStore
  • JA-SIG uPortal JIRA UP-1578 Refactored setAggregatedLayout to adhere to foreign key constraints (which were added to the database)
  • JA-SIG uPortal JIRA UP-1577 and optimized it (performing smart updates).

• org/jasig/portal/channels/groupsmanager/Utility
  • Changed apache xml serializer imports to org.apache.xml.serialize.* JA-SIG uPortal JIRA UP-1579

• org/jasig/portal/channels/portlet/CPortletAdapter
  • Changed portlet window ID to channel's uid instead of the subscribe ID JA-SIG uPortal JIRA UP-1580

• org/jasig/portal/layout/channels/CSkinSelector
  • Made skin path configurable JA-SIG uPortal JIRA UP-1581

• org/jasig/portal/security/provider/AuthorizationImpl
  • Remove excessive object creation by properly synchronizing around the principalCache, rather than cloning and replacing.
    JA-SIG uPortal JIRA UP-1582