Page Comparison

[03:44:55 EDT(-0400)] * mad (n=chatzill@pcit-8752.hig.se) has joined ##uportal
[08:35:20 EDT(-0400)] * athena7 (n=athena7@adsl-99-149-83-32.dsl.wlfrct.sbcglobal.net) has joined ##uportal
[09:55:03 EDT(-0400)] * colinclark (n=colin@142.150.154.101) has joined ##uportal
[10:00:37 EDT(-0400)] * EricDalquist (n=dalquist@bohemia.doit.wisc.edu) has joined ##uportal
[10:01:13 EDT(-0400)] <EricDalquist> can people here log into the wiki?
[10:39:25 EDT(-0400)] * bulloche (n=bulloche@134.250.4.77) has joined ##uportal
[10:47:48 EDT(-0400)] <grimesp> someone upgraded the wiki to 2.9... nice
[10:48:50 EDT(-0400)] <EricDalquist> yeah
[10:48:55 EDT(-0400)] <EricDalquist> did that yesterday
[10:49:02 EDT(-0400)] <EricDalquist> too hot to be doing anything outside
[10:57:21 EDT(-0400)] <athena7> looks good to me EricDalquist
[10:57:36 EDT(-0400)] <EricDalquist> if it stays hot jira might get moved too
[11:02:46 EDT(-0400)] <athena7> gotcha
[11:19:00 EDT(-0400)] * holdorph (n=holdorph@wsip-98-174-242-39.ph.ph.cox.net) has joined ##uportal
[11:36:57 EDT(-0400)] * lennar1 (n=sparhk@wsip-98-174-242-39.ph.ph.cox.net) has joined ##uportal
[11:48:41 EDT(-0400)] * colinclark (n=colin@142.150.154.101) has joined ##uportal
[11:50:09 EDT(-0400)] * anastasiac (n=team@142.150.154.160) has joined ##uportal
[11:59:58 EDT(-0400)] * michelled (n=team@142.150.154.197) has joined ##uportal
[12:43:06 EDT(-0400)] * awills (n=awills@wsip-98-174-242-39.ph.ph.cox.net) has joined ##uportal
[13:19:17 EDT(-0400)] * colinclark (n=colin@142.150.154.101) has joined ##uportal
[13:23:37 EDT(-0400)] <EricDalquist> awills: that last patch removed the last bit of dynamic cached content
[13:23:54 EDT(-0400)] <EricDalquist> working on an export of our prod snapshot and only have about 50 keys in the cache
[13:24:08 EDT(-0400)] <EricDalquist> it used to constantly grow due to all of those simple little xpaths
[13:24:14 EDT(-0400)] <awills> only 50 keys? that's pretty cool
[13:24:23 EDT(-0400)] <EricDalquist> not a big deal CPU wise but lots of wasted memory
[13:24:45 EDT(-0400)] <EricDalquist> helps that the change to do that was dead simple due to that callback interface
[13:25:30 EDT(-0400)] <awills> yeah XPath variables is a tidy little enhancement... it's a good match
[13:25:34 EDT(-0400)] <EricDalquist> also I think have have the export layout working so that it will export entity prefs for a user that has no layout
[13:25:38 EDT(-0400)] <EricDalquist> yeah
[13:25:46 EDT(-0400)] <EricDalquist> fits with the features of jexl/groovy
[13:26:04 EDT(-0400)] <awills> yeah, exactly
[13:26:31 EDT(-0400)] <EricDalquist> oh and other random performance note
[13:26:44 EDT(-0400)] <EricDalquist> found a few sites that say jexl is 5x or more faster than groovy
[13:27:06 EDT(-0400)] <EricDalquist> so you probably want to lean toward that for simple expression stuff with tests and attributes
[13:27:10 EDT(-0400)] <awills> it's as though all these technologies have the basic TaskRequest concept, just different types and method sigs... all you have to do is get what you have into the expected format and get out of the way
[13:27:26 EDT(-0400)] <awills> ok, that makes sense
[13:27:44 EDT(-0400)] <awills> it might nead rexamining w/ groovy 1.6
[13:27:57 EDT(-0400)] <EricDalquist> true
[14:28:41 EDT(-0400)] <EricDalquist> actually awills you could do the same callback pattern with scripts
[14:29:01 EDT(-0400)] <EricDalquist> instead of copying all of the attributes into a Bindings each time crn runs groovy/js/tc
[14:29:04 EDT(-0400)] <EricDalquist> etc*
[14:29:15 EDT(-0400)] <EricDalquist> you could have a custom Bindings impl that just does a callback into the request
[14:32:51 EDT(-0400)] <awills> that's true... that sounds good... would save some of that copying
[14:33:22 EDT(-0400)] <EricDalquist> not sure if it really matters at all
[14:33:33 EDT(-0400)] <EricDalquist> but if you have a javax.script heavy crn script
[14:33:46 EDT(-0400)] <EricDalquist> you could spend a fair bit of time just copying data
[14:36:02 EDT(-0400)] <awills> i'm inclined to think it makes sense... I bet Bindings is a pretty simple interface
[14:36:15 EDT(-0400)] <EricDalquist> JexlContext would be the final place to do the same thing
[14:36:21 EDT(-0400)] <EricDalquist> yeah
[14:36:28 EDT(-0400)] <EricDalquist> Bindings is really just a simplified Map
[14:36:38 EDT(-0400)] <EricDalquist> you could even wrap another Bindings interface for the set* methods
[14:36:50 EDT(-0400)] <EricDalquist> and default the wrapped instance to SimpleBindings
[14:37:08 EDT(-0400)] <EricDalquist> then you can be lazy and only implement the get* logic as a simple override
[14:41:13 EDT(-0400)] <EricDalquist> just did an all-layouts export for 146198 users, 18344 of them with layouts in just over 15 minutes
[14:48:18 EDT(-0400)] <awills> that's nuts... i didn't think there was that kind of improvement to be had
[14:48:32 EDT(-0400)] <EricDalquist> another problem we ran into
[14:48:40 EDT(-0400)] <EricDalquist> uPortal tables are terribly indexed
[14:48:48 EDT(-0400)] <EricDalquist> UP_USER had no index or primary key
[14:48:49 EDT(-0400)] <awills> well yes
[14:49:04 EDT(-0400)] <EricDalquist> along with problems in up_struct_param
[14:49:15 EDT(-0400)] <EricDalquist> so we had some db tuning to do
[14:49:32 EDT(-0400)] <awills> my understanding was that these things were difficult to do properly w/ dbloader
[14:49:32 EDT(-0400)] <EricDalquist> still have some SQL performance tweaking the user export section
[14:49:43 EDT(-0400)] <EricDalquist> I bet we can do a full 150k user export in under 20 minutes
[14:49:49 EDT(-0400)] <EricDalquist> some of them are
[14:49:55 EDT(-0400)] <EricDalquist> but not the ones we're missing
[14:50:03 EDT(-0400)] <awills> alright then
[14:50:08 EDT(-0400)] <EricDalquist> foreign keys are the big thing that dbloader would be hard pressed to do
[14:50:16 EDT(-0400)] <EricDalquist> but a lot of it is just lack of info in tables.xml
[14:50:26 EDT(-0400)] <EricDalquist> 2.6 and 3.0 are better than 2.5 was
[14:50:29 EDT(-0400)] <EricDalquist> but they still need work
[14:50:43 EDT(-0400)] <awills> if you can export in 20 min, it seems you'll be able to turn the whole thing around in under 3 hrs
[14:51:01 EDT(-0400)] <awills> when is your date?
[14:51:20 EDT(-0400)] <EricDalquist> Oct 21
[14:51:27 EDT(-0400)] <EricDalquist> have a lot of other work that needs to be done to
[14:51:33 EDT(-0400)] <awills> i bet
[14:51:43 EDT(-0400)] <EricDalquist> but this makes us feel a lot better about the timeslot we have alloted
[14:53:58 EDT(-0400)] <awills> if it were we doing what you're doing, I'd feel pretty good about the fact that I just had to identify how each type of operation can be improved and do so one place, rather than chase all these things down in each small part of each operation that's done on each portal object type
[14:54:04 EDT(-0400)] <awills> *me
[14:54:44 EDT(-0400)] <EricDalquist>