Statistics, Metrics, Reporting for uPortal UnConf08

Owned by Jim Helwig
Last updated: Oct 08, 2008 by Andrew PetroVersion comment

What do folks want to talk about?

what is different with out of the box between uPortal 2.5, 2.6, 3.0?

Watch this space!

UW-Madison has extended the stats recorder.

Denison put a hook in to record session stats.

How can you extend what stats are recorded?

what stats do people generally report or want to report?

  • Logins - who, time, host name, group/role membership
  • Portlet access or use (how long)
    • What does portlet use mean? Rendering, certainly
    • Note: a portlet be by just your eyeballs
    • usage is sometimes impossible to determine
    • You can sometimes force folks to 'acknowledge' use (e.g., MS 'remove this from desktop?')
    • You could automatically minimize portlets that have not been used
    • You can't track usage if it isn't through the portal (e.g., links/bookmarks); may need to correlate logs on another server
    • May want to track what action took place; this is portlet-specific
    • It would be nice to have libraries or some way to easily add the recording of portlet-specific events/actions
  • Page/tab views; subtabs (when, which tab, how long)
  • layout modifications (removing, adding, rearranging)

what are the metrics that help us measure success?

  • Need to work with content owners to figure out what the questions are.
  • Need to translate raw metrics into aggregates that help answer the questions, meaningful to stakeholders
  • Stakeholders set measures of success

What is the audience of the reports? How are they being used?

  •  Service providers
    • is my stuff being used
    • who is using my stuff? Is my target audience using it?
  • IT staff
    • performance
    • capacity
    • browsers/platform (for focusing testing)

Later in maturity of product

  • progress towards goals

Effort is required for this. Make sure that reports are going to be used. What are the questions? What are they going to do with the data?

Usage stats? Fragments?

What are people using to generate reports?

What tools are being using? Commercial? Open Source?

  • NetInSight for web analytics - using apache logs, and other server access logs
  • NextBrick - OpenDocs portal analyzer - maybe supports uPortal. For product information, checkout: http://www.nextbrick.com/products/reportingAnalysis/uPortalMetrics/produPortalMetrics.htm
  • Milicken/StFrancis - JSP with simple reports on the data they have collected; van look for various event codes
  • Log4J can be used to uPortal logs
  • You can define custom events and event handlers via stats recorder
  • Two types of metrics: general that can be handled by framework; portlet-specific
  • Need to consider how to handle business logic specific events
  • Could hook into container logging or develop a best practice for general web app logging
  • Portlet-specific logs or put in a singe log
  • If it is portlet specific logs, you still want to be able to correlate with the info that the framework has
  • UW-Madison uses Informatica to handle some aggregation of metrics and then Hyperion to do reporting, but we are scrapping this and developing some custom aggregation and reporting based on RRDB
  • Recording in a database can facilitate correlation
  • Flat text files are easy to grep, access, secure,manage

UW-Madison uses a jMeter script that logs into each of the MyWisc portal nodes and checks each for functionality, emailing and logging to a database if it detects a failure. The script and configuration file is available.

You'll need the latest version of jMeter for this to work.

How are folks storing stats?

Logging to a flatfile, database

Milicken/St Francis - custom table, custom code