uPortal IRC Logs-2012-05-09

Owned by Apereo Infrastructure
Last updated: May 09, 2012
6 min read

[14:30:15 CDT(-0500)] <b-sure> hello uPortal devs: during a rebuild of the user data in our dev up4 portal, I''ve managed for it to be configured that when I log in , I see the orange fragment administration banner across the top several times. Do you know what permission I have assigned to me that would cause this to show up?

[14:36:45 CDT(-0500)] <EricDalquist> the banner is likely in every fragment

[14:36:51 CDT(-0500)] <EricDalquist> and you have extended headers on

[14:37:05 CDT(-0500)] <EricDalquist> you need to modify your fragment layout files

[14:37:14 CDT(-0500)] <b-sure> oh ok.

[14:37:15 CDT(-0500)] <EricDalquist> set the header and footer folders as hidden="true"

[14:37:18 CDT(-0500)] <EricDalquist> then reimport

[14:37:26 CDT(-0500)] <b-sure> ok thanks. I'll give that a try.

[14:37:27 CDT(-0500)] <EricDalquist> just the fragment files

[14:37:30 CDT(-0500)] <b-sure> ok

[14:40:42 CDT(-0500)] <b-sure> Hi Eric. I also notice that I have links to edit things like "Edit Column 1 permissions" and "Edit page permissions"

[14:41:33 CDT(-0500)] <b-sure> is that related to the extended headers being on?

[14:50:24 CDT(-0500)] <EricDalquist> um

[14:50:29 CDT(-0500)] <EricDalquist> I think so

[14:50:37 CDT(-0500)] <EricDalquist> I can't remember for sure though

[14:51:05 CDT(-0500)] <athena1> those show up in fragment administration mode so you can lock down nodes

[14:51:14 CDT(-0500)] <athena1> that's how you set a portlet to be unremovable, etc.

[14:51:25 CDT(-0500)] <EricDalquist> yeah

[14:51:27 CDT(-0500)] <b-sure> yeah. It seems like when I log in Im in the fragment admin mode some how

[14:51:31 CDT(-0500)] <EricDalquist> but I remember seeing that

[14:51:39 CDT(-0500)] <EricDalquist> when we did our data upgrade

[14:51:46 CDT(-0500)] <EricDalquist> I thjink fixing the header include/exclude resolves it

[14:53:15 CDT(-0500)] <b-sure> ok thanks EricDalquist. that fixed it for me.

[14:53:29 CDT(-0500)] <EricDalquist> great

[14:55:10 CDT(-0500)] <b-sure> I ended up optioning to delete all of the users files and start fresh. with everyone having the default layout. Hoping we don't get a huge user backlash.

[14:55:39 CDT(-0500)] <EricDalquist> no luck getting layout migration working?

[14:56:43 CDT(-0500)] <b-sure> well we would have used the imported data where only about 100 layout files didn't import, but the issue is that we coulndn't tell how many of the success would break on login.

[14:57:29 CDT(-0500)] <b-sure> so the file imported cleanly , but it would still break for a user when they logged in.

[14:58:07 CDT(-0500)] <EricDalquist> you never tracked down the cause of the breakage?

[14:58:14 CDT(-0500)] <b-sure> drew_wills noticed something about the exported data file from 3.2.5 so maybe that has to do with it.

[15:00:02 CDT(-0500)] <b-sure> maybe I should make a ticket from the pastebin I put here a couple days ago. Not sure if any of the developers will have time to look at this again.

[15:00:28 CDT(-0500)] <EricDalquist> right, I don't really have time to look into it

[15:00:37 CDT(-0500)] <b-sure> were pushing to upgrade so we can start load testing again.

[15:00:45 CDT(-0500)] <EricDalquist> do you have any support resources (unicon or other) that can assist with solving the problem?

[15:01:40 CDT(-0500)] <b-sure> well I think we are out of support for this year (ends in June) and we want to upgrade before that so I think were just cutting our losses. everything else with the upgrade is pretty ready to go.

[15:02:07 CDT(-0500)] <EricDalquist> (sad)

[15:02:19 CDT(-0500)] <b-sure> EricDalquist, do you at madison load test the portal?

[15:02:27 CDT(-0500)] <EricDalquist> yeah

[15:02:33 CDT(-0500)] <EricDalquist> we have some jmeter scripts we use to do the load testing

[15:02:44 CDT(-0500)] <EricDalquist> nothing fancy

[15:02:45 CDT(-0500)] <b-sure> we want to see if we can get some expectations of load we achieve

[15:02:49 CDT(-0500)] <EricDalquist> just login, visit tabs

[15:02:52 CDT(-0500)] <b-sure> we use grinder

[15:02:53 CDT(-0500)] <EricDalquist> logout

[15:03:00 CDT(-0500)] <b-sure> same kind of thing.

[15:03:08 CDT(-0500)] <EricDalquist> well, we just load test based on passed metrics

[15:03:27 CDT(-0500)] <EricDalquist> do you have any data about logins per hour or tab views per hour you can go off of?

[15:03:43 CDT(-0500)] <b-sure> this fall we are forcing students to go though the portal to complete an administrative process so were bracing for a load spike

[15:04:32 CDT(-0500)] <b-sure> well on our latest tests, 32bit server 2 gig memory 3 tomcat nodes, we got to 200 concurrent users before tomcat broke

[15:04:41 CDT(-0500)] <b-sure> this was a few months ago.

[15:05:00 CDT(-0500)] <b-sure> we have since upgraded to 64bit and 4g memory per node but haven't run tests on these yet

[15:05:12 CDT(-0500)] <EricDalquist> there were a lot of improvements in 4.0.5 and 4.0.4

[15:05:18 CDT(-0500)] <EricDalquist> we see sub-second page render times

[15:05:21 CDT(-0500)] <b-sure> we are on the latest release

[15:05:30 CDT(-0500)] <EricDalquist> with 500-600 active sessions on each of 4 nodes

[15:05:55 CDT(-0500)] <b-sure> thats good to hear. do you use 64bit and how much memory per node?

[15:06:25 CDT(-0500)] <EricDalquist> https://wiki.jasig.org/display/UPC/JVM+Configurations

[15:06:31 CDT(-0500)] <EricDalquist> that is pretty much up to date

[15:06:49 CDT(-0500)] <b-sure> so we've always had a bottleneck of about 5+ seconds during logins. but after that, page load times are average of less than 3 seconds.

[15:07:08 CDT(-0500)] <EricDalquist> sounds like a potential issue with attribute retrieval

[15:07:09 CDT(-0500)] <b-sure> seems like uPortal does a lot of front loading of data for the user

[15:07:15 CDT(-0500)] <EricDalquist> do you do much in person directory

[15:07:21 CDT(-0500)] <b-sure> yeah plus we use grouper,

[15:07:34 CDT(-0500)] <EricDalquist> yeah

[15:07:41 CDT(-0500)] <EricDalquist> so to figure out what to show the user

[15:07:49 CDT(-0500)] <EricDalquist> the portal has to know all the user's attributes and groups

[15:07:55 CDT(-0500)] <b-sure> we do get a majority of user data from remote request attributer

[15:07:58 CDT(-0500)] <EricDalquist> so it hits the attributes and groups stores HARD on login

[15:08:14 CDT(-0500)] <EricDalquist> so if those are slow or not caching data appropriately things will be slow

[15:08:29 CDT(-0500)] <b-sure> but we also have an ldap attribute source configured so that we can user swap

[15:09:12 CDT(-0500)] <b-sure> yeah. I don't think it is easy or feasible to try to cache all of the data from grouper in the portal.

[15:10:23 CDT(-0500)] <EricDalquist> well you don't need to cache it all

[15:10:49 CDT(-0500)] <b-sure> that group service makes a lot of calls to the backing data store (grouper in our case) to get all of the groups for the user. if any of the groups are named generically like "student" or "staff" that returns a ton of results

[15:11:22 CDT(-0500)] <EricDalquist> but a reasonable cache config could look like:

[15:11:22 CDT(-0500)] <EricDalquist> cache 1000 results of "is member of" check for "UserX, GroupY" with a TimeToIdle of 60 seconds

[15:11:35 CDT(-0500)] <EricDalquist> that way if the same check gets made a lot during login

[15:11:37 CDT(-0500)] <EricDalquist> you only get it once

[15:11:43 CDT(-0500)] <EricDalquist> and it falls out of the cache quickly

[15:11:53 CDT(-0500)] <EricDalquist> there are lots of caches in uportal like thjat

[15:11:58 CDT(-0500)] <EricDalquist> there to help with short loads

[15:12:05 CDT(-0500)] <EricDalquist> not to keep all data in memory forever

[15:12:17 CDT(-0500)] <b-sure> yeah. I tried to do something like that with the grouper code we submitted, I just don't think it is working very well yet

[15:12:22 CDT(-0500)] <EricDalquist> I'm not familiar with the grouper group service

[15:12:33 CDT(-0500)] <EricDalquist> is it using ehcache?

[15:12:41 CDT(-0500)] <b-sure> it isn't

[15:12:49 CDT(-0500)] <EricDalquist> ah

[15:13:01 CDT(-0500)] <EricDalquist> yeah I'd recommend looking at refactoring that to use ehcache

[15:13:03 CDT(-0500)] <b-sure> I think it just uses a hasmap to store all of the groups it collects

[15:13:14 CDT(-0500)] <EricDalquist> if that is something you're interested in I can share a custom group store we wrote

[15:13:19 CDT(-0500)] <EricDalquist> that is wired up in spring

[15:13:23 CDT(-0500)] <EricDalquist> and does caching via ehcache instances

[15:13:24 CDT(-0500)] <b-sure> is ehcache available to the code there as a sevrice?

[15:13:27 CDT(-0500)] <b-sure> oh ok

[15:15:27 CDT(-0500)] <b-sure> yeah I would definately like to refine how groups are retrieved from the grouper service to try to limit the number of calls

[15:16:53 CDT(-0500)] <EricDalquist> ok

[15:16:59 CDT(-0500)] <EricDalquist> I'll get it bundled up

[15:17:04 CDT(-0500)] <EricDalquist> and posted tomorrow

[15:19:13 CDT(-0500)] <b-sure> ok. Thanks Eric.