RPMs to Manage Deployments

Contribution from Tom Freestone at BYU

Continuous Delivery is a software development pattern for automating the deployment of enhancements/bugs to production with little risk and minimal overhead. The principle of Continuous Delivery can be implemented using an RPM. This tutorial demonstrates how to configure and deploy a uPortal install using an RPM built by Maven. Local configuration and life-cycle is customizable using Maven build profiles, filters, overlays and the RPM plugin.

Assumptions

We are assuming that the RPM artifact contains everything needed to run uPortal. This tutorial creates a single RPM artifact that includes both the uPortal WAR, the portlet WARs, the Tomcat install and the Apache configuration. This technique can easily be extended to support separate RPMs. This tutorial assumes a general familiarity with how to install uPortal, Tomcat, Linux, etc. You may want to familiarize yourself with the uPortal install process. This tutorial is uses uPortal 4.x but is applicable to previous releases. For previous releases, you will need to add directories for the various filter properties files. You can create them as they exist in uPortal 4.x.

Add a New Maven Module Project.

In the uPortal parent pom.xml, add a new module that will create the RPM artifact. You can name it PortalRPM.

<modules>
   <module>uportal-war</module>
   <module>uportal-ear</module>
   <module>uportal-portlets-overlay</module>
   <module>bootstrap</module>
   <module>uportal-search-api</module>
   <module>PortalRPM</module>
</modules>

Overlay Artifacts

Artifacts may overwritten or overlaid using the Maven War Plugin. The Portlet Overlays directory utilizes this technique with portlets. However, other artifacts can also be overlaid. For instance, you may wish to change tomcat settings during various different software life-cycles. To overlay tomcat, first download Tomcat from Apache. For production environments, you may wish to delete the docs and examples applications that come with the default Tomcat download. Change the file extension to .tgz and upload to your Maven repo. Tomcat can then be used as a Maven dependency.

Create Maven Filters and Resources

Create properties files for each of your various software life-cycle phases such as local, development, QA, RC and production. You define system properties you want to filtered within the properties file. In uPortal, properties files are stored ${uPortalRoot}/filters. For consistency, the properties files are named after the life-cycle phase (e.g. prod.properties). The following is an example properties file with various different properties. You can add additional properties to these files.

noQuickstartData=true
## Database Connection Settings (Uncomment the Maven Filters section in rdbm.properties)
environment.build.hibernate.connection.driver_class=oracle.jdbc.OracleDriver
environment.build.hibernate.connection.url=jdbc:oracle:thin:@ldap://oid.someuniversity.edu:389/somedatabase,cn=OracleContext,dc=someuniversity,dc=edu
environment.build.hibernate.connection.username=someuser
environment.build.hibernate.connection.password=somepassword
environment.build.hibernate.dialect=org.hibernate.dialect.Oracle10gDialect
environment.build.dbcp.validation.query=select sysdate from dual
# uPortal server configuration properties
environment.build.uportal.server=my-uPortal.someuniversity.edu
environment.build.uportal.protocol=https
environment.build.uportal.context=/uPortal
environment.build.uportal.email.fromAddress=portal@someuniversity.edu

# CAS server configuration properties
environment.build.cas.server=somecas.someuniversity.edu
environment.build.cas.protocol=https

environment.build.log.rootLevel=INFO
# Directory to place portal and portlet log files into.
environment.build.log.logfileDirectory=${catalina.base}/logs
# Assume a DailyRollingFileAppender is used. Set the pattern to daily log-file roll-overs.
# Can also set to hourly, weekly, etc.
# Can also set to hourly, weekly, etc.  Use yyyy-MM-dd-HH for hourly.
# See http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/DailyRollingFileAppender.html
environment.build.log.rollingLogFileDatePattern=yyyy-MM-dd
# Pattern to specify format of each log file entry.  See http://logging.apache.org/log4j/1.2/apidocs/index.html.
environment.build.log.layoutConversionPattern=%5p [%t] %c{2}.[%x] %d{ISO8601} - %m%n

# LDAP server configuration properties
environment.build.ldap.url=ldap://ldap.someuniversity.edu:389
environment.build.ldap.username=uid=someuser,ou=someone,o=someuniversity.edu
environment.build.ldap.password=someuser
environment.build.ldap.userbase=ou=people,o=someuniversity.edu
#environment.build.ldap.resourcebase=
environment.build.ldap.groupbase=ou=groups,o=someuniversity.edu
environment.build.ldap.uidattribute=uid
environment.build.ldap.displayName=displayName

# Example local variables
portalValidationQuery=select sysdate from dual

statsDatabaseDriver=oracle.jdbc.OracleDriver
statsDatabaseURL=jdbc:oracle:thin:@ldap://oid.someuniversity.edu:389/somedatabase,cn=OracleContext,dc=someuniversity,dc=edu
statsDatabaseUser=someuser
statsDatabasePassword=somepassword
statsDatabaseValidationQuery=select sysdate from dual
statsDatabaseHibernateDialect=org.hibernate.dialect.Oracle10gDialect

portletDatabaseURL=jdbc:oracle:thin:@ldap://oid.someuniversity.edu:389/somedatabase,cn=OracleContext,dc=someuniversity,dc=edu
portletDatabaseUser=someuser
portletDatabasePassword=somepassword
portletDatabaseDriver=oracle.jdbc.OracleDriver
portletDatabaseValidationQuery=select sysdate from dual
portletDatabaseHibernateDialect=org.hibernate.dialect.Oracle10gDialect
...

Define build profiles in uPortal parent pom.xml. Build profiles defined in the parent POM are global and are inherited by each module. Modules can override properties in their local POM. You may wish to add variables which are build profile specific. The following is an example build profile where we define the Oracle jdbc jar and version of tomcat as properties.

<profiles>
...
        <profile>
            <id>prod</id>
            <properties>
                <env>prod</env>
                <jdbc.groupId>com.oracle</jdbc.groupId>
                <jdbc.artifactId>ojdbc6</jdbc.artifactId>
                <jdbc.version>11.2.0.3.0</jdbc.version>
                <tomcat.version>6.0.37</tomcat.version>
            </properties>
        </profile>
...
</profiles>

You can filter global resources in the uPortal parent pom.xml using the Maven Resource Plugin. In uPortal 4.x, the Maven Resource Plugin is enabled by default. In earlier uPortal releases, the Maven Resource Plugin will need to be enabled. If you want to filter web resources (e.g. web.xml) you will need to use the Maven War Plugin in conjunction with the Maven Resource Plugin.

Next, you will need to define resources. Resources specify the files and the directories you want to substitute variables in using the Maven Resource Plugin. Variables often change between build-profiles. For instance, we wish to filter our tomcat configuration than we define resources in our PortalRPM pom.xml. Here is a sample resource definition to filter tomcat configuration:

<build>
...
  <resources>
    <resource>
      <directory>${project.basedir}/src/main/config/tomcat</directory>
      <filtering>true</filtering>
      <includes>
        <include>**/*.xml</include>
       <include>**/*.properties</include>
      </includes>
  </resource>            
...
 </resources>
...
 </build>

In those resources that you have selected for filtering, you can now add filter variables that can be substituted in at build-time. Filter variables specified by ${} are defined in your property files and build profiles definitions.

For instance, you may wish change database configuration between build profiles for differing tomcat installs. The following is a fragment from the conf/server.xml of Tomcat in the RPM project.

... 
<GlobalNamingResources>
    <!-- Editable user database that can also be used by
         UserDatabaseRealm to authenticate users
    -->
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
      ...
        <Resource auth="Container"
                  name="PortalDatabase"
                  type="javax.sql.DataSource"
                  driverClassName="${databaseDriver}"
                  maxIdle="5"
                  maxWait="-1"
                  poolPreparedStatements="true"
                  removeAbandoned="true"
                  removeAbandonedTimeout="120"
                  logAbandoned="true"
                  validationQuery="${validationQuery}"
                  url="${portalSecureURL}"
                  initalSize="8"
                  maxActive="16"/>
        <Resource auth="Container"
                  name="PortalStatsDatabase"
                  type="javax.sql.DataSource"
                  driverClassName="${databaseDriver}"
                  maxIdle="5"
                  maxWait="-1"
                  poolPreparedStatements="true"
                  removeAbandoned="true"
                  removeAbandonedTimeout="120"
                  logAbandoned="true"
                  validationQuery="${validationQuery}"
                  url="${portalStatsSecureURL}"
                  initalSize="8"
                  maxActive="16"/>
        <Resource auth="Container"
                  name="PortletDatabase"
                  type="javax.sql.DataSource"
                  driverClassName="${databaseDriver}"
                  maxIdle="5"
                  maxWait="-1"
                  poolPreparedStatements="true"
                  removeAbandoned="true"
                  removeAbandonedTimeout="120"
                  logAbandoned="true"
                  validationQuery="${validationQuery}"
                  url="${portletSecureURL}"
                  initalSize="8"
                  maxActive="16"/>
        ...
</GlobalNamingResources>
...

Enable RPM Maven Plugin

The Maven RPM Plugin allows you to produce a RPM as build artifact. Here is how to enable the Maven RPM Plugin with the PortalRPM pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>rpm-maven-plugin</artifactId>
    <version>2.0.1</version>
    <extensions>true</extensions>
</plugin>

In order to build a RPM, you will need to configure various settings within the configuration block. Please refer to the documentation for details (http://mojo.codehaus.org/rpm-maven-plugin/params.html).

The uPortal RPM depends on various other RPMS's such as jdk, httpd etc. The requires block defines those other RPM dependencies in which uPortal needs. Here is a sample configuration of other RPM's needed to run uPortal.

...
<configuration>
...
  <requires>
      <require>jdk</require>
      <require>httpd</require>
      <require>httpd-devel</require>
      <require>mod_ssl</require>
      <require>tomcat-native</require>
  </requires>
...
</configuration>
...

Maven dependencies can be defined in the configuration block. Previously, we uploaded tomcat into our Maven repo. In the artifactItems block, we can define Tomcat as a dependency. We can also specify the jdbc driver we wish to use with Tomcat. The following example illustrates this.

... 
<configuration>
...
<artifactItems>
...
 <artifactItem>
        <groupId>edu.byu.images</groupId>
        <artifactId>apache-tomcat</artifactId>
        <type>tgz</type>
        <version>${tomcat.version}</version>
        <destFileName>tomcat.tar.gz</destFileName>
        </artifactItem>
     <artifactItem>
         <groupId>${jdbc.groupId}</groupId>
         <artifactId>${jdbc.artifactId}</artifactId>
         <version>${jdbc.version}</version>
         <type>jar</type>
         <outputDirectory>${dbJar}</outputDirectory>
      </artifactItem>
      ...  
</artifactItems>
...
</configuration>
...

A RPM needs to define where artifacts are installed. The Maven RPM Plugin does this mapping directives. Here is a sample of a mapping directive that defines where tomcat dependency (previous example) will be installed.

<configuration>
   ... 
   <mappings>
   ...
      <mapping>
          <directory>/opt/tomcat</directory>
          <sources>
              <source>
                  <location>
                      ${project.basedir}/target/images/apache-tomcat-${tomcat.version}
                  </location>
              </source>
          </sources>
      </mapping>
   ...
  </mappings>
... 
</configuration>

Build the RPM

uPortal's is built using ant tasks. To build an RPM, we create a new ant task

<target name="deploy-rpm" depends="rpmBuild, deploy-ear" description="Creates RPM to deploy to servers">
    <antcall target="mvn">
        <param name="pomDir" value="PortalRPM"/>
        <param name="goal" value="package"/>
    </antcall>    
</target>
<target name="rpmBuild">
    <copy file="build.properties.prod" tofile="build.properties"/>
</target>

By default, uPortal installs to the ${server.base}/webapps. However, when an rpm is built, this will not be the case. You need to create separate build.properties files. Typically, you will not want a local rpm install of uPortal. Here is an example build.properties for building RPM's.

##### Replace server.home with the location of you want RPM to built #####
# path to tomcat binaries
server.home=/data/work/build

# path to tomcat servlet container (defaults to same directory as the tomcat binaries)
server.base=${server.home}

# path to webapps directory (defaults to /webapps under the tomcat path)
server.webapps=${server.base}/webapps

# If the mvn executable isn't on your path you can set the location here or set it on your path.
#maven.home=


######## Deployment Configuration ##########
# The following apply to deployPortletApp, deploy-war, and deploy-ear targets

# Should the deployment tools extract the WARs when copying them into the servlet container 
extractWars=true

# Should the existing webapp be removed before deploying the new webapp
removeExisting=true

# The following applies to the clean and deploy-ear targets.
# Should the shared library location have all existing files removed when clean is run.
# NOTE: Tomcat 6.0 does not include any libraries in shared/lib by default so as long as
#       all required libraries are declared in the uportal-ear it is ok to set this to true.
cleanShared=true

# Prompt the user before running ant tasks that could modify the database
prodPrompt=true

Sample Code

Sample of the Portal RPM project that is used at BYU is located here

Additional Resources

Spring 2011 Jasig Conference Rapid Deployment Models for uPortal

Having problems with these instructions?

Please send us feedback at uportal-user@lists.ja-sig.org