This wiki page is for collaborating on a proposed overhaul of layout support in Import/Export. It's late March, 2009, and uPortal 3.1.0 has just been released. The changes under consideration are dramatic, and would likely be introduced in uPortal 3.2.0, if adopted.

Overhauling Import/Export of Layouts

The Import/Export subsystem was added to uPortal in December, 2007. Since then many institutions have come to rely on it, especially in migrating to uPortal 3 since Import/Export is the only supported mechanism. These tools have evolved since 2007 and currently offer pretty comprehensive support for uPortal data. Nevertheless, there are still some feature gaps in Import/Export that cause real trouble; and most of these are in the area of layouts.

Thorny Layout-Related Issues

These issues don't have easy fixes in the current Import/Export tech for layouts.

Import/Export Does Not Support All DLM Layout Functions

This issue is documented in UP-2358 and partially in UP-2255 and again partially in UP-2247.

The scripts do not properly handle all cases of exporting and importing DLM data. The following are missing after export: deleted tabs, reordered tabs, renamed tabs, and possibly (probably) others.

Import/Export Does Not Support Arbitrary Layers of Nesting in Layouts (e.g. Subtabs)

This issue is documented in UP-2188.

This issue is important for institutions with subtabs, particularly those who may want to translate their MyVT-based theme to uP 3.

Import/Export Does Not Support Multiple User Profiles

This issue is documented in UP-2174.

In uP 2 the profile was a part of the user document (1 / user); in uP 3 it's in the layout document (still 1 / user). This issue is important for institutions looking at adding a mobile theme to their portal.

Import/Export Only Supports DLM

This is less of an issue now than it once was. It would have been nice to provide a smooth transition from ALM onto DLM, but that ship has largely sailed by now. Supporting additional IUserLayoutManager implementations may prove useful, nevertheless, if new implementations emerge in the future.

Proposals

The following changes represent a major re-tooling of layout support in Import/Export. Taken together, they solve all of the issues listed above. These changes are the sort that should go into a full minor release (3.2), though they could be "backported" (carefully) by organizations that need them for importing right away.

Move User Profiles to A Separate Document

Right now Import/Export supports only a single layout and profile per user. uPortal arguably supports multiple layouts and profiles, though these muscles are only rarely flexed and we expect there are lurking defects in these features. (Jen Bourey did some recent investigation, she may know the most.)

If we create a separate .profile document, we can at least support multiple profiles (using a single layout). This enhancement is probably all that's needed for Import/Export to support portals with an additional theme for mobile devices.

Each profile document would be assumed to reference layout #1 (much like now).

Move Much Responsibility for Layout Export to the IUserLayoutStore Implementations

The real cause of the issues listed above is the fact that Import/Export completely bypasses the IUserLayoutStore object(s) and goes directly for the database. This approach obligates Import/Export to duplicate all the logic and domain knowledge they embody. It also obligates Import/Export to keep up with these technologies when they evolve. It's tough to grasp all the ways that RDBMDistributedLayoutStore stores and transforms data. And guess what: I (Andrew Wills) didn't.

It helps to remember that Import/Export was originally developed as an "overlay" to uPortal: meaning that it was designed to enhance uPortal through the introduction of new files only, without altering any existing files. This constraint led me not consider some options that might have been smoother; options we can consider now.

We should add import() and export() methods to the IUserLayoutStore interface. In the export() method, each IUserLayoutStore implementation would be responsible for returning a complete, "externalized" (free of database identifiers) XML representation of the specified user's layout. They would be free, furthermore, to base this representation on their own, internal model for what a layout looks like. They should also remove as much "noise" – things like channel publishing parameters, which are handled in .channel documents – from this representation as possible. In the import() method, each IUserLayoutStore would be responsible for consuming the same XML schema it puts out.

This would be a major change to the XML format of layout documents, and would require us to define a new Cernunnos script to import them at classpath:/org/jasig/portal/io/import-layout_v3-2.crn. XML from existing versions can continue to be handled by the import-layout_v3-0.crn file (the most recent) without changes. In this way, the current XML will continue to be supported as well as it ever was. Alternately, we can explore transforming the earlier XML into the new format; but no amount of transformation will provide the data elements that are missing.

Provide An "-export" Branch

The first two proposals leave something important off the table: previous versions of uPortal don't have these capabilities. You wouldn't be able to migrate uPortal versions with these fixes unless and until someone needs to migrate from uP 3.2 onto a version even later than that.

We can't do anything about XML that's already been exported; it's already missing the information we're after. But we could possibly provide a "hybrid" branch of uPortal source code designed to connect to an earlier database version (e.g. uP 3.0.x) and produce XML in the new style on export. Eric Dalquist has previously followed this pattern for exporting uP 2 data to uP 3 format.

Status

I created UP-2406 in JIRA to track this effort; there is a patch attached.