Production Setup

This page contains example config files and directions for setting up a production portal using a combination of Linux/Apache/Java/Tomcat/uPortal.

Right now it is very much a work in progress. Please, feel free to add/correct and delete content.

Baseline Production Apache Config

TODO:

example server.xml with jk on 8009 and with http on 8080 disabled

These instructions cover the following programs and versions:

uPortal 2.4.x
Redhat Linux AS/ES
Apache 2.x
Tomcat 5.0.28
Java 1.4.2_05

Specific configuration

Java

Install j2sdk-1_4_2_05-linux-i586-rpm.bin

Simlink the sdk's directory to /usr/java. This will make java upgrades easier because the jre can be upgraded by moving the simlink:

simlink Java sdk directory


mv j2sdk1.4.2_05 /usr/java/
ln -s j2sdk1.4.2_05/ java

Set JAVA_HOME


echo export JAVA_HOME="/usr/java/java" >> /etc/profile

Perform the same two commands at the command prompt now to set Java home.


echo JAVA_HOME

Apache

Add the following to /etc/httpd/conf/httpd.conf

httpd.conf


...
Include /usr/local/tomcat/conf/mod_jk.conf

mod_jk.conf


#### Load Mod_Jk ################################

<IfModule !mod_jk.so>
        LoadModule jk_module modules/mod_jk.so
</IfModule>

#Tomcat Security Section
<LocationMatch "/WEB-INF/">
  AllowOverride None
  deny from all
</LocationMatch>

<LocationMatch "/META-INF/">
  AllowOverride None
  deny from all
</LocationMatch>

#### Virtual Hosts ###############################

Listen 0.0.0.0:80
Listen 0.0.0.0:443

NameVirtualHost uportal-vip.foo.edu:80
NameVirtualHost uportal-vip.foo.edu:443

# The non SSL part of the site just redirects to the SSL port
<VirtualHost uportal.foo.edu:80>
        ServerName uportal.foo.edu
        DocumentRoot /usr/local/tomcat/webapps/ROOT
        ErrorLog /etc/httpd/logs/error_log
        CustomLog /etc/httpd/logs/access_log common
        JkMount /*.jsp wrkr
        DirectoryIndex index.jsp
</VirtualHost>

# All access to uPortal is done over SSL
<VirtualHost uportal.foo.edu:443>
        ServerName uportal.foo.edu
        DocumentRoot /usr/local/tomcat/webapps/ROOT
        ErrorLog /etc/httpd/logs/error_log
        CustomLog /etc/httpd/logs/access_log common

        JkMount /*.jsp wrkr
        JkMount /uPortal wrkr
        JkMount /uPortal/*.jsp wrkr
        JkMount /uPortal/*.uP wrkr
        JkMount /uPortal/Authentication wrkr
        JkMount /uPortal/Login wrkr
        JkMount /uPortal/Logout wrkr
        JkMount /uPortal/*.jws wrkr
        JkMount /uPortal/services/* wrkr
        JkMount /uPortal/servlet/* wrkr

        DirectoryIndex index.jsp
        SSLEngine on
        SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
        SSLCertificateFile /etc/httpd/conf/ssl/uportal.foo.edu.crt
        SSLCertificateKeyFile /etc/httpd/conf/ssl/uportal.foo.edu.key
        <Files ~ "\.(cgi|php|shtml|phtml|php3?)$">
           SSLOptions +StdEnvVars
        </Files>

        Alias /uPortal  "/usr/local/tomcat/webapps/uPortal"

        <Directory "/usr/local/tomcat/webapps/ROOT">
            AllowOverride None
            Order allow,deny
            Allow from all
            DirectoryIndex index.jsp
        </Directory>
        <Directory "/usr/local/tomcat/webapps/uPortal">
            AllowOverride None
            Order allow,deny
            Allow from all
        </Directory>


</VirtualHost>

#### Tomcat Log File Setup ######################

JkWorkersFile "/usr/local/tomcat/conf/workers.properties"
JkLogFile "/usr/local/tomcat/logs/mod_jk.log"
JkLogLevel info
JkLogStampFormat "[%a %b %d %H: %M: %S %Y]"

JK Connector

Commands to install and compile mod_jk


tar -zxvf jakarta-tomcat-connectors-jk-1.2.src-current.tar.gz
mv jakarta-tomcat-connectors-jk-1.2.5-src /usr/src
cd /usr/src/jakarta-tomcat-connectors-jk-1.2.5-src/jk/native
./buildconf.sh
./configure -with-apxs=/usr/sbin/apxs
make
mv apache-2.0/mod_jk.so /etc/httpd/modules/

/usr/local/tomcat/conf/workers.properties example


worker.list=wrkr
worker.wrkr.port=8009
worker.wrkr.host=localhost
worker.wrkr.type=ajp13
worker.wrkr.cachesize=10
worker.wrkr.cache_timeout=600
worker.wrkr.socket_timeout=300

Tomcat


useradd tomcat
passwd tomcat (what should we type for the password)
tar -zxvf jakarta-tomcat-5.0.28.tar.gz
mv jakarta-tomcat-5.0.28 /usr/local/
ln -s jakarta-tomcat-5.0.28 tomcat
chown -R tomcat:tomcat jakarta-tomcat-5.0.28

The following script is called when tomcat is shutdown or restarted. This script captures important diagnostic information that will allow you to in many cases diagnose portal problems such as slow external resources, database problems, synchronization/concurrency issues and slow code.

/usr/local/bin/portal-status


#!/bin/bash
date >> /usr/local/tomcat/logs/portal-status.log
# save the garbage colection log
cp /usr/local/tomcat/logs/tomcat_gc.log /usr/local/tomcat/logs/tomcat_gc_`date +%y%m%d_%H%M%S`.log
# run netstat
netstat -e >> /usr/local/tomcat/logs/portal-status.log

# run top to get memory and process info
top -n 1 -b >> /usr/local/tomcat/logs/portal-status.log

# apache status tells us what apache was up to
/usr/sbin/apachectl status >> /usr/local/tomcat/logs/portal-status.log

# send a SIGQUIT to the java process to force it to dump a threaddump to the catalina.out log
PID=`ps -AH | pgrep java |head -n 1`
kill -3 $PID

/etc/init.d/tomcat


#!/bin/sh
#
# Startup script for Tomcat, the Apache Servlet Engine
#
# Tomcat name :)
TOMCAT_PROG=tomcat

# if TOMCAT_USER is not set, use tomcat like Apache HTTP server
if [ -z "$TOMCAT_USER" ]; then
 TOMCAT_USER="tomcat"
fi

RETVAL=0

export JAVA_HOME="/usr/java/java/"
export CATALINA_OPTS="-Xms256m -Xmx512m -Xloggc:/usr/local/tomcat/logs/tomcat_gc.log -Djava.awt.headless=true -Dsun.net.client.defaultReadTimeout=600000 -Dsun.net.client.defaultConnectTimeout=120000"
export CATALINA_HOME=/usr/local/tomcat
export JPDA_ADDRESS=8000

# start and stop functions
start() {
    echo -n "Starting tomcat: "
    ulimit -v 1048576
#    chown -R $TOMCAT_USER:$TOMCAT_USER /usr/local/tomcat/*
#    chown -R $TOMCAT_USER:$TOMCAT_USER /home/tomcat/*
    su $TOMCAT_USER -c "${CATALINA_HOME}/bin/catalina.sh start ${CATALINA_OPTIONS}"
    RETVAL=$?
    echo
    [ $RETVAL = 0 ] && touch /var/lock/subsys/tomcat
    return $RETVAL
}

stop() {
    echo -n "Stopping tomcat: "
    ###
    echo "Dumping portal-status to portal-status.log. Dumping JVM thread dump to catalina.out."
    /usr/local/bin/portal-status
    sleep 10
    ###
    su $TOMCAT_USER -c "${CATALINA_HOME}/bin/catalina.sh stop"
    RETVAL=$?
    echo
    [ $RETVAL = 0 ] && rm -f /var/lock/subsys/tomcat /var/run/tomcat.pid
    rm -rf /usr/local/tomcat/work/*
}

# See how we were called.
case "$1" in
  start)
        start
        ;;
  stop)
        stop
        ;;
  restart)
        stop
        # Ugly hack
        # We should really make sure tomcat
        # is stopped before leaving stop
        sleep 2
        start
        ;;
  *)
        echo "Usage: $0 {start|stop|restart}"
        exit 1
esac

exit $RETVAL

Copy your JDBC jar to your /usr/local/tomcat/common/lib directory. For example we copy ojdbc14.jar (Oracle jdbc thin client) and jtds-x.x.x.jar (MS SQLServer jdbc driver) because we connect to Oracle and MS SQL using JNDI and DBCP.

ROOT Context

/usr/local/tomcat/webapps/ROOT/index.jsp


<%@ page errorPage="error.jsp" %>
<%
  // we need to do the following instead if tomcat is behind mod_jk with ssl to force ssl
  response.sendRedirect("https://"+request.getServerName() + "/uPortal/render.userLayoutRootNode
.uP");
%>

In the Tomcat 5.x line you will need to edit the web.xml in ROOT/WEB-INF as well, as it maps requests for /index.jsp to the index_jsp servlet. Comment out the <servlet-mapping> and add this code inside the <web-app> block.


<welcome-file-list>
        <welcome-file>
            index.jsp
        </welcome-file>
</welcome-file-list>

Also, I placed a robots.txt file in ROOT:

/usr/local/tomcat/webapps/ROOT/robots.txt


User-agent: *
Disallow: /

web.xml

Configure the default session timeout to your desired session timeout.

Portal Settings

In portal.properties set the following to avoid the mixed-content warning when hosting uPortal over ssl:

properties/portal.properties


...
org.jasig.portal.serialize.ProxyWriter.resource_proxy_enabled=on
org.jasig.portal.serialize.ProxyWriter.resource_proxy_rewrite_prefix=/uPortal/servlet/ProxyServlet/


...

Production Installation Notes:

We have found in our uPortal performance tests completed fall 2003 that leaving on debug information in the class files DOES NOT slow down the portal significantly. The debug information is very important to have when dianosing a stack trace as without it there are no line numbers printed in the stack trace.