This page is for collecting ideas on how to ease database upgrades in uPortal.

Exporting and importing domain-specific artifacts

The way Confluence handles upgrades is very impressive. Thoughts along those lines.

The way to ease database upgrades in uPortal is to stop thinking about databases and instead think about importing, exporting, and upgrading versions of the domain model. That is, move the problem up out of the DDL layer and into the application / service layer. Import, export, and upgrade are features of the application. Any given deployed version of uPortal needs to be able to talk to its own DAO implementations which access its own backing DDL / RDMS layer. Each version also needs to be able to import and export domain-specific, implementation-agnostic representations of the content and configuration of the portal.

The upgrade path becomes: make the current, deployed version of uPortal export its content into the files. Bring up the new version of uPortal against a fresh database. Instruct the new version to import the files that were exported by the old version.

Versioned XML schemas

There need to be XML schemas defining a particular version of the definition of how to describe a piece of the uPortal domain in XML.

For instance, there would be version 1.0 of a schema for describing channel types. That schema might define an XML structure conveying a human-friendly name for the type, a section for the "channel publishing document" content describing the workflow for configuring the publications and subscriptions of the channel, etc.

There would also be a version 1.0 of a schema for describing a published channel. That schema might define an XML structure conveying the channel title, display name (for selecting it at subscribe time, for instance), an fname key, the name of the channel type, channel parameters and metadata about whether the end user can override them, categories for the channel, groups authorized to subscribe to the channel.

Some version of uPortal uses version 1.0 of these schemas, let's say.

Suppose uPortal adds a new feature: a longer, more globally unique key for channel types and versioning of channel types. Version 1.1 of the schema for describing channel types adds two attributes: one for the version of the channel type, and one for the key to the channel type. Version 1.1 of the schema for describing published channels drops the reference to the channel type name and replaces it with a channel type key, version pair.

In domain language

The XML is about being a portal. It describes pieces of the portal.

The XML isn't about Databases

The XML schemas do not describe tables, columns, or foreign keys. The approach described here isn't that of DbLoader.

The XML isn't about Java

The XML schemas do not describe Java classes. The approach described here isn't that of Spring beans XML files.

The XML is about Portals

The XML schemas would descibe portal content in terms of portal concepts.

Exporting and Importing

Exporting

So, going with the example. From the live uPortal that uses Schemas version 1.0, a deployer would use the administrative UI to export the channel type definitions and published channels (along with other data as part of a larger export, perhaps). Out comes XML, possible zipped up into a .zip.

The deployer brings online an improved uPortal which uses the new schema. Deployer wants to import the channels and channel types from the old uPortal. Uses the import UI to instruct the uPortal to import from the .zip.

Importing

The new uPortal, pursuant to backwards compatibility, knows how to parse and use not just the latest version of the schemas, but previous versions as well. As it parses the .zip content, it discovers the XML files and reads their schema declarations. It sees schema version 1.0 and uses the appropriate parser / interpretter.

The old schema doesn't specify the new data for channel types, so the uPortal sets the guids to be the same as the channel type names and it sets the version of each channel type to be 1.0. The old schema for published channels expresses the channel type association in terms of channel type name - the import process would recognize this and translate it to a modern association in terms of channel type guid and version.

The underlying DDL probably changed significantly - the published channel table has different columns with different foreign key constraints. But since the process went through the intermediary of strongly typed XML, dealing with hot-changing database schemas isn't necessary.