uPortal3 Adoption - University of Wisconsin-Madison

Owned by Jim Helwig
Last updated: Feb 20, 2009Version comment

High Level Project Plan

Initial development and testing

  1. Get metrics on code diff regarding UW framework customizations
  2. Apply uPortal 3 code to my-predev3
    • Custom layout preferences
    • Config mode
    • API layer
    • Skin/theme
    • Build scripts
  3. Migrate UW framework customizations to my-predev3
  4. Export QA layout and import into my-predev
  5. Test to ensure QA version of portlets work in my-predev
  6. Duplicate my-predev database for my-predev3 and perform conversion (export/import)
  7. Test QA version of portlets in my-predev3
  8. Load test my-predev3
  9. Developers and service providers test in my-predev3
  10. Estimate upgrade work for each environment

Environment upgrades

  1. Upgrade my-predev
    • Java, apache, mod_jk, openssl patching
    • Add new ant, mavin and Tomcat
    • Create new database
    • Build and install uPortal3, local mods, portlets
    • Full export of existing portal instance
    • Full import into new database
  2. Test and let any dust settle in my-predev (2 weeks)
  3. Upgrade my-dev and my-test (leave my-uptest, my-updev as is)
    • See steps above
  4. Test and let any dust settle in my-dev/my-test (2 weeks)
  5. Upgrade my-qa (leave my-upqa as two servers)
    • See steps above
  6. Regression and load test my-qa (1 week)
  7. Upgrade a few production servers
    • See steps above
  8. Test
  9. Switch to new servers
    • Full export of current production portal database
    • Diff with previous export
    • Import diff
    • Point Layer 4 VIP to uP3 servers instead of uP2 servers
  10. Upgrade remaining uP3 production servers

Framework Modifications

Build Scripts

Create an additional uw-build.xml which will be an include in the uPortal build.xml. The uw-build.xml file will encapsulate the uw-* ant targets. While working on this script all other ant scripts in the build system should be reviewed for compatibility and functionality.

  • Use CATALINA_HOME if set over the value in build.properties
  • uw-tomcat-clean-all target
    • depnds on uw-tomcat-clean-app and uw-tomcat-clean-fwk
  • uw-tomcat-clean-app
    • remove all webapps from tomcat
    • remove all jars from shared lib
    • remove shared/classes
    • clean out conf/Catalina/localhost
  • uw-tomcat-clean-fwk
    • Remove the portal webapp
    • Remove the portal context config file
  • JSP compilation
  • uwcompile target
    • compiles all uportal source without the stop-on-error behavior
  • uw-pubchan target
    • allows channel publishing from any directory
  • installPortlets target
    • calls dist target on ../portlets/build.xml
    • calls deployPortletApp on directory ../portlets/dist/
  • cleanWorkDirectory target
    • deletes out tomcat/work
  • db-stats target
    • sets up the statistics tables

Configuration

Spring contexts

    • personLookup
    • xpathPool
    • ldapContext

dlm.xml

Just copy this over whole-sale

LDAP

Translate ldap.xml into ldapContext.xml

Person Directory

Update config to work with Person Directory 1.1.0

portal.properties

  • extended stats config
  • groups manager ldap search limit

rdbm.properties

Review if these should be configured as global (server wide) JNDI data sources. Tomcat docs for global jndi Along with this will be updating the uPortal context.xml file to source these JNDI datasources in as needed.

Groups

Copy config files over as is, look into merging flatGroupStoreContext.xml with the main portal application context

worker.properties

Configuration of a prefs worker for the custom layout preferences UI

Portal API Abstraction Layer

Configuration

Merge configuration and initialization into portal's application context from services.xml

Person Directory

This will need to be updated to use the new APIs, also it should be looked into to just implement the shared IPersonAttributeDao class.

Groups and Permissions

No changes have been made to the GAP code so this should pretty much copy over as is unless we try to switch to stand-alone GAP during the upgrade.

LDAP Server Access

The wrapper code will likely need to be tweaked to work with the Spring-LDAP APIs

JDBC DataSource Access

The wrapper code will likely need to be tweaked to use global JNDI resources or spring context datasource resources.

Swapper Portlets

ID Swapper

The portlet code should copy over fairly easily, may want to look at cleaning up to use latest Spring apis and such. Will need to review Login/Logout servlet modifications that enable this behavior. There is a swapRedirect property that also needs to be configured in security.properties to enable this.

Attribute Swapper

The portlet could should copy over fairly easily, may want to look at cleaning up to use latest Spring apis and such.

Person Lookup

This code should just copy as is. We should look at incorporating this into the Person Directory project.

Extended Stats Recording

These will need to be refactored to work with the more event based stats framework in 3.0. The object model should stay the same though.

Misc

XPath Pooling Support

This needs to be reviewed to see if it is still needed in 3.0. Once all of the other framework changes have been made and performance testing is happening we can review to see if XPath compilation is still a bottleneck.

RemoteUserSecurityFilter

Code can just be copied and configured in web.xml, this is to help catch redirect loops and such.

Custom PAGS testers

Just copy these over, look to include them in the stand-along GAP project.

Multi-valued channel attributes

These exist solely to allow multi-valued portlet preferences in chanpubs. The real solution here will be to extend the ChannelDefinition object to have a <preferences> section that is specific to portlet preferences.

UW Theme & Skin

The theme and skin files need to be slightly modified to work on uPortal 3.0

Lessons Learned

  • Don't go first; It was very beneficial that JHU flushed out performance issues earlier this summer. Thanks JHU!!!
  • Don't assume that a quiet Help Desk means acceptable performance
  • Monitor early and often, continuously if feasible
  • Expect increasing load to expose issues; this includes an increasing number of unique users, not just concurrent users
  • If feasible, be prepared to flip back to previous install
  • Always keep the Help Desk informed
  • Using an IM chatroom is a great way to keep everyone in the loop
  • Don't declare victory before experiencing peak load
  • Ensure developers get early and repeated announcements of available test environments
  • Regularly check with developers on testing progress
  • Ensure test accounts and/or test cases are published
  • Upgrading par t of the cluster and doing a quick swap made the transition practically transparent
  • Identify and receive approval for one or two alternative upgrade dates in advance
  • Put a moratorium on portlet changes - make it clear that regular application updates will not be installed the same day as a major upgrade
  • Document and disseminate information regarding UI changes; even though the portal "looked the same," the implementation was different.
  • Use open source (thanks, uPortal!) and have key developers on site (thanks, Eric!)