Event Persistence Configuration

Owned by Eric Dalquist
Mar 25, 2015
4 min read

Data Storage Considerations

The raw and aggregate events databases can take a considerable amount of space. The default configuration is fairly conservative, only retaining 1 hour worth of raw and only aggregating a limited set of time intervals and groups. To make space management and upgrades easier it is recommended to place the raw and aggregate events tables in their own schemas or databases.

DataSource Configurations

To better manage the storage and performance requirements of the raw event and aggregate event data sets these can be moved into their own databases.

Step 1: To configure separate JDBC connection information for these data sets edit uportal-war/src/main/resources/properties/rdbm.properties and change the RawEvents and AggrEvents database configuration.

 

rdbm.properties
##### uPortal Raw Events DB
RawEventsJdbcDriver=${environment.build.hibernate.connection.driver_class}
RawEventsJdbcUrl=${environment.build.hibernate.connection.url}
RawEventsJdbcUser=${environment.build.hibernate.connection.username}
RawEventsJdbcPassword=${environment.build.hibernate.connection.password}

##### uPortal Aggregate Events DB
AggrEventsJdbcDriver=${environment.build.hibernate.connection.driver_class}
AggrEventsJdbcUrl=${environment.build.hibernate.connection.url}
AggrEventsJdbcUser=${environment.build.hibernate.connection.username}
AggrEventsJdbcPassword=${environment.build.hibernate.connection.password}

 

Step 2: After modifying the rdbm.properties file to store the events to a different database,  perform either of the following:

  1. ant initdb  (BEWARE: performing this command will re-initialize your entire database, use with caution. )
  2. ant db-hibernate-raw-events and/or ant db-hibernate-aggr-events 

Persistent Event Filtering

The default configuration results in all PortalEvents being persisted to the RawEvents database. To exclude certain events, filters can be configured in the listenerEventFilters list in uportal-war/src/main/resources/properties/contexts/eventHandlerContext.xml. There is provided support for filtering events as supported or ignored by type and by the user the event was generated for.  

listenerEventFilters list in eventHandlerContext.xml
<!-- 
 | List of event filters to apply to all event listeners that handle newly fired events.
 +-->
<util:list id="listenerEventFilters">
    <!-- Can be used to filter events based on event type using an instanceof check -->
    <bean class="org.jasig.portal.spring.context.ClassApplicationEventFilter">
        <property name="supportedEvents">
            <list>
                <value>org.jasig.portal.events.PortalEvent</value>
            </list>
        </property>
        <!-- Can configure a list of events to ignore based on event type using an instanceof check. For example
             if you are using Google Analytics, you can ignore most PortletExecutionEvents (retain
             PortletActionExecutionEvent for logging Search Events for the search terms) and some other Events.
             You may want to retain login, logout, layout changes, Portlet hung, and tenant events.
             Note that the PortletExecutionEvents include information about time to execute the portlet event
             (action, render, event, and resource events) which could be helpful to report on times that
             individual portlets are taking to do their operations.  At the moment though there are currently
             no reports shipping in uPortal to aggregate and report on that information (you could write it and
             contribute it though!).  It is an example of information that Google Analytics would not currently
             report on.
        <property name="ignoredEvents">
            <list>
                <value>org.jasig.portal.events.PortletEventExecutionEvent</value>
                <!- - <value>org.jasig.portal.events.PortletActionExecutionEvent</value> retain for Search terms - ->
                <value>org.jasig.portal.events.PortletRenderExecutionEvent</value>
                <value>org.jasig.portal.events.PortletRenderHeaderExecutionEvent</value>
                <value>org.jasig.portal.events.PortletResourceExecutionEvent</value>
                <value>org.jasig.portal.events.PortalRenderEvent</value>
            </list>
        </property>
        -->
    </bean>
    <!-- Can be used to filter events based on who generated the event
    <bean class="org.jasig.portal.events.handlers.UserFilteringPortalEventHandler">
        <property name="ignoredUserNamePatterns">
            <list>
                <value>user\..*</value>
                <value>vptest-user.*</value>
            </list>
        </property>
        <property name="supportGuest" value="true" />
    </bean>
     -->
</util:list>

Using ClassApplicationEventFilter you can specify the types of PortalEvent you want to include or ignore.  The image below shows the list of events as of the time of the image capture.

Google Analytics Reports overlap with portal events

Google Analytics reports (see Google Analytics) provide extensive reports and overlaps quite a bit with uPortal's internal event reports.  The recommended strategy is to configure Google Analytics and use it as the primary reporting mechanism.  However, there are still advantages to retaining some uPortal events for local reporting.  When using Google Analytics, the recommended list of classes to add to the ignoredEvents property of the listenerEventFilters is noted in the commented-out section above.

Purging Persistent Events

The Event Aggregation process also handles purging of aggregated events. The default configuration is to purge raw event data 1 hour after it has been aggregated. This window size can be changed by setting the org.jasig.portal.events.aggr.PortalEventPurgerImpl.purgeDelay property in uportal-war/src/main/resources/properties/portal.properties