uPortal IRC Logs-2009-09-02

Owned by Apereo Infrastructure
Last updated: Sept 02, 2009Version comment
3 min read

[06:35:28 EDT(-0400)] * higmad (n=chatzill@pcit-8752.HIG.SE) has joined ##uportal
[08:56:36 EDT(-0400)] * EricDalquist (n=dalquist@bohemia.doit.wisc.edu) has joined ##uportal
[09:05:30 EDT(-0400)] * athena (n=athena@adsl-75-58-127-15.dsl.wlfrct.sbcglobal.net) has joined ##uportal
[10:07:42 EDT(-0400)] * higmad_ (n=chatzill@130.243.6.30) has joined ##uportal
[11:01:52 EDT(-0400)] * lennard1 (n=sparhk@ip68-98-56-21.ph.ph.cox.net) has left ##uportal
[11:06:50 EDT(-0400)] * holdorph (n=holdorph@wsip-72-215-204-133.ph.ph.cox.net) has joined ##uportal
[11:34:18 EDT(-0400)] * lennard1 (n=sparhk@wsip-72-215-204-133.ph.ph.cox.net) has joined ##uportal
[11:49:05 EDT(-0400)] <lennard1> Eric?
[11:49:22 EDT(-0400)] <EricDalquist> busy with local issue ... hopefully resovled soon
[11:49:27 EDT(-0400)] <lennard1> k
[11:49:51 EDT(-0400)] <lennard1> Will throw some info up her for you and whoever else is interested.
[11:51:40 EDT(-0400)] <lennard1> Last night Pearson's HE Portal that is running on uPortal 2.6 experienced some issues around logout. These problems eventually became crippling and a restart did not/would not resolve the problem.
[11:51:47 EDT(-0400)] <lennard1> The table at the heart of the problem is the table - UP_ENTITY_CACHE_INVALIDATION.  At the time of the incident the table had 750K rows.
[11:51:58 EDT(-0400)] <lennard1> The problem query was the following: DELETE FROM   UP_ENTITY_CACHE_INVALIDATION       WHERE   INVALIDATION_TIME < :1
[11:52:24 EDT(-0400)] <lennard1> So... we truncated the table, and there was much rejoicing(smile)
[11:52:47 EDT(-0400)] <lennard1> Everything appears to be running fine now. Was just wondering about a few things.
[11:52:56 EDT(-0400)] <lennard1> 1) Have you ever heard of this issue before?
[11:53:37 EDT(-0400)] <lennard1> 2) Does that table exist in uPortal 3.x, and if so... would this problem and its resolution be worth letting the mailing list know about?
[11:54:06 EDT(-0400)] <lennard1> The table field - INVALIDATION_TIME – was not indexed.  Thus the above delete query, which happens for every single logout, likely was going through a full table scan to delete rows.
[11:55:17 EDT(-0400)] <lennard1> Honestly though... that table should never have grown so large. I suspect the system which pearson uses for SSO was having issues... and that, that was likely the reason the table grew until the point that it became a problem in and of itself.
[12:30:05 EDT(-0400)] <EricDalquist> so, I have never seen that
[12:30:18 EDT(-0400)] <EricDalquist> but we don't run with the portal.properties multi-server flag turned on
[12:30:29 EDT(-0400)] <EricDalquist> which is the only time that table is used I believe
[12:30:35 EDT(-0400)] <EricDalquist> it is still there in 3.x
[12:30:46 EDT(-0400)] <EricDalquist> only used by the GAP framework for multi-server consistency
[12:30:58 EDT(-0400)] <EricDalquist> the rest of the portal now relies on ehcache's multi-server support
[12:33:31 EDT(-0400)] <lennard1> You run multiple servers though, right?
[12:33:41 EDT(-0400)] <EricDalquist> yes
[12:33:54 EDT(-0400)] <EricDalquist> but we don't make changes to groups or permissions without server restarts
[12:34:03 EDT(-0400)] <EricDalquist> which is the only thing that table helps with
[12:34:59 EDT(-0400)] <lennard1> pretty much same situation for us as well.
[12:35:02 EDT(-0400)] <EricDalquist> honestly in 3.x since GAP is backed by ehcache as well using the distributed invalidation caching options may negate the need for those tables
[12:35:16 EDT(-0400)] <EricDalquist> then you shouldn't need to run with multiserver set to true
[12:36:33 EDT(-0400)] <lennard1> org.jasig.portal.concurrency.multiServer=true
[12:36:34 EDT(-0400)] <lennard1> org.jasig.portal.concurrency.clockTolerance=5000
[12:36:37 EDT(-0400)] <lennard1> yeah... we have it on
[12:53:27 EDT(-0400)] * colinclark (n=colin@bas2-toronto09-1176444611.dsl.bell.ca) has joined ##uportal
[12:55:05 EDT(-0400)] * colinclark (n=colin@bas2-toronto09-1176444611.dsl.bell.ca) has joined ##uportal
[13:05:22 EDT(-0400)] <lennard1> Thanks for the info Eric(smile)
[13:50:14 EDT(-0400)] * colinclark (n=colin@bas2-toronto09-1176444611.dsl.bell.ca) has joined ##uportal
[18:27:37 EDT(-0400)] * holdorph (n=holdorph@wsip-72-215-204-133.ph.ph.cox.net) has joined ##uportal
[19:04:27 EDT(-0400)] * colinclark (n=colin@bas2-toronto09-1176444611.dsl.bell.ca) has joined ##uportal
[20:38:44 EDT(-0400)] * lennard1 (n=sparhk@wsip-72-215-204-133.ph.ph.cox.net) has left ##uportal
[20:59:06 EDT(-0400)] * colinclark (n=colin@bas2-toronto09-1176444611.dsl.bell.ca) has joined ##uportal