uPortal IRC Logs-2008-08-29
- Apereo Infrastructure
[06:25:36 EDT(-0400)] * mad (n=chatzill@pcit-6254.HIG.SE) has joined ##uportal <awills> i've used <return> mostly w/ $ , but I'm planning to add a public Object ScriptRunner.evaluate() method too <EricDalquist> <attribute key="Attributes.CACHE">$ </attribute> <EricDalquist> though it could just be a phrase and you have to use $ to use cernunnos for the factory <awills> yeah, it would... and it would leave open the possibility to implement a fancy Phrase that does just what you want, instead of using $ <EricDalquist> <cache key="Attributes.NODE" cache-key="$ " factory="$ "> <EricDalquist> <echo-ln>Cache Status: $ /$ </echo-ln> <EricDalquist> if I do <attribute key="Attributes.CACHE">$ </attribute> <EricDalquist> and I think something like <sql-datasource driver="$ " url="$ " username="$ " password="$ "> would end up creating multiple db pools <EricDalquist> $ ? <awills> what about $ from before the <xslt>? <awills> oh yes...$ <EricDalquist> $
[09:19:39 EDT(-0400)] * mad_ (n=chatzill@pcit-6254.hig.se) has joined ##uportal
[10:03:29 EDT(-0400)] * EiNZTEiN (n=einztein@205.241.143.3) has joined ##uportal
[10:04:01 EDT(-0400)] * EiNZTEiN_ (n=einztein@205.241.143.3) has joined ##uportal
[10:17:05 EDT(-0400)] * EricDalquist (n=dalquist@bohemia.doit.wisc.edu) has joined ##uportal
[10:32:40 EDT(-0400)] * michelled (n=team@142.150.154.197) has joined ##uportal
[10:59:08 EDT(-0400)] * lennar1 (n=sparhk@wsip-98-174-242-39.ph.ph.cox.net) has joined ##uportal
[11:24:31 EDT(-0400)] * holdorph (n=holdorph@wsip-98-174-242-39.ph.ph.cox.net) has joined ##uportal
[12:09:31 EDT(-0400)] * awills (n=awills@wsip-98-174-242-39.ph.ph.cox.net) has joined ##uportal
[12:24:42 EDT(-0400)] <EricDalquist> hey awills, is there any way to have a task's sub-tasks return some value?
[12:26:06 EDT(-0400)] <awills> if i understand correctly, yes... with <return>
[12:26:13 EDT(-0400)] <EricDalquist> ah yeah
[12:26:35 EDT(-0400)] <EricDalquist> I think I'm going to add a <cache key=""></cache> task
[12:26:50 EDT(-0400)] <EricDalquist> in re-factoring the layout export to deal with the dlm node IDs I'm seeing quite a use for that
[12:27:13 EDT(-0400)] <EricDalquist> so I don't have to constantly re-build the limited-layout doms
[12:27:25 EDT(-0400)]
[12:27:40 EDT(-0400)] <EricDalquist> cool
[12:28:32 EDT(-0400)] <awills> that would be cool... also, fwiw, I would think a <cache> task would go in a ...cache pkg 
[12:29:38 EDT(-0400)] <EricDalquist> oh and an easy way to fix possible memory problems with cache-all:
[12:29:39 EDT(-0400)] <awills> is there a danger of "ORA-01000: maximum open cursors exceeded" on 10g if the default OPEN_CURSORS isn't bumped?
[12:29:41 EDT(-0400)]
[12:29:50 EDT(-0400)] <EricDalquist> no idea
[12:29:51 EDT(-0400)] <awills> yes, definately
[12:31:21 EDT(-0400)] <awills> docs i'm reading suggest the default value is 50... trying to guess whether ORA-1000 means (1) the value's too low, or (2) some code is leaking db objects
[12:31:55 EDT(-0400)] <EricDalquist> I believe it would be leaking ResultSet objects
[12:32:21 EDT(-0400)] <awills> yeah, there's a chance of that
[12:50:06 EDT(-0400)] <awills> Eric – this morning I have word from Chris Doyle (who independently tested the most recent SmartLdap patch on JHU portal) that it works 100%... are we good to go on that?
[12:50:19 EDT(-0400)] <EricDalquist> yup
[12:50:46 EDT(-0400)] <awills> excellent, good to clear at least 1 thing off my plate :?
[12:53:44 EDT(-0400)] <EricDalquist> hrm
[12:53:50 EDT(-0400)] <EricDalquist> so this cache task is a bit more complex
[12:54:02 EDT(-0400)] <EricDalquist> I'm thinking it will need to look like
[12:54:31 EDT(-0400)] <EricDalquist> <cache key="" threadsafe="t/f"><factory></factory><subtaks></subtasks></cache>
[12:57:02 EDT(-0400)] <awills> factory's just a class name right? could instead be <cache key="" threadsafe="t/f" factory=""><do stuff.../></cache>
[12:57:10 EDT(-0400)] <EricDalquist> no
[12:57:23 EDT(-0400)] <EricDalquist> factory list like subtasks
[12:57:25 EDT(-0400)] <awills> yeah i was thinking that one over... i follow
[12:57:57 EDT(-0400)]
[12:58:01 EDT(-0400)] <EricDalquist> that would be simpler
[12:58:26 EDT(-0400)] <EricDalquist> because the crn phrase would only get executed if I actually called factoryPhrase.evaluate(req, res) right?
[12:58:39 EDT(-0400)]
[13:00:42 EDT(-0400)] * grimesp (n=grimesp@134.250.4.177) has joined ##uportal
[13:01:00 EDT(-0400)] <EricDalquist> so now I have
[13:01:22 EDT(-0400)] <EricDalquist> <cache key="" cache-key="" threadsafe="t/f" factory=""></cache>
[13:01:37 EDT(-0400)] <EricDalquist> key is the name of the attribute the cached obj would be bound to
[13:11:02 EDT(-0400)] * jessm (n=Jess@c-76-19-199-61.hsd1.ma.comcast.net) has joined ##uportal
[13:32:54 EDT(-0400)] * holdorph (n=holdorph@wsip-98-174-242-39.ph.ph.cox.net) has joined ##uportal
[13:57:28 EDT(-0400)] <Sememmon> dorph!
[13:59:40 EDT(-0400)] * holdorph smacks Sememmon with a large trout.
[14:00:04 EDT(-0400)] <Sememmon> _
[14:34:13 EDT(-0400)] <EricDalquist> gah
[14:34:22 EDT(-0400)] <EricDalquist> database I'm testing the export against just went down :po
[14:34:25 EDT(-0400)] <EricDalquist> makes it hard to test
[14:34:50 EDT(-0400)] <holdorph> ouch, yeah, tends to.
[14:34:56 EDT(-0400)] <EricDalquist> on the plus side I do have the cache task working:
[14:34:57 EDT(-0400)]
[14:36:33 EDT(-0400)] <EricDalquist> I wonder which is faster for simple stuff like that ... groovy or jexl 
[15:22:19 EDT(-0400)] <EricDalquist> hrm
[15:22:24 EDT(-0400)] <EricDalquist> awills: a crn question for you
[15:22:48 EDT(-0400)] <EricDalquist> http://uportal.pastebin.com/m7bc5d80f
[15:22:56 EDT(-0400)] <EricDalquist> the echo prints the string I want
[15:23:05 EDT(-0400)] <EricDalquist> but in the with-attribute that calls that script I just get null
[15:23:10 EDT(-0400)] <EricDalquist> if the otherwise executes I get the right value
[15:32:45 EDT(-0400)] * michelled (n=team@142.150.154.197) has left ##uportal
[16:09:23 EDT(-0400)] <awills> what does lookup-dlm-noderef.crn look like?
[16:10:22 EDT(-0400)] <EricDalquist> http://uportal.pastebin.com/m25c68e83
[16:10:59 EDT(-0400)] <EricDalquist> so the problem ended up being using a CernunnosPhrase in with-attribute clobbers the RETURN attribute that the nested <return> uses
[16:12:00 EDT(-0400)] <awills> yeah that sounds like a distinct possibility... i'll look at the patch here
[16:12:32 EDT(-0400)] <EricDalquist> the patch checks if there is an existing return attribute, grabs it to a local, runs the phrase, then puts the original back
[16:15:41 EDT(-0400)] <EricDalquist> hrm ... I wonder if a dlm:positionSet can ever exist under a tab instead of a column ....
[16:17:29 EDT(-0400)] <awills> Here's another take on the RETURN_VALUE issue – the RuntimePhraseDecorator doesn't do what the RuntimeTaskDecorator does: decorate the TaskRequest. If it did, I believe this issue would go away
[16:18:09 EDT(-0400)] <EricDalquist> that it would
[16:18:13 EDT(-0400)] <EricDalquist> perhaps that is the solution?
[16:18:17 EDT(-0400)] <awills> though I'd have to do some quick thinking to feel confident the solution wouldn't interfere with something else
[16:18:26 EDT(-0400)] <awills> i kind of thinkk it is, eric
[16:19:19 EDT(-0400)] <awills> phrases are basicaally just like tasks w/ a different method sig i think, in my head, i thought all the same mechanics were occuring
[16:19:42 EDT(-0400)] <awills> i need to step away for a sec, brb
[16:19:48 EDT(-0400)] <EricDalquist> no problem
[16:20:05 EDT(-0400)] <EricDalquist> yeah I was confused when debugging because I was expecting the same decoration mechanics for phrases
[16:20:27 EDT(-0400)] <EricDalquist> otherwise you get this case where a phrase can clobber request/response attributes from the parent
[16:35:30 EDT(-0400)] <awills> yeah
[16:35:43 EDT(-0400)] <awills> just got back, did you try it yet?
[16:37:17 EDT(-0400)] <EricDalquist> nope
[16:39:46 EDT(-0400)] <EricDalquist> trying it now
[16:40:33 EDT(-0400)] <EricDalquist> looks like that works
[16:40:46 EDT(-0400)] <EricDalquist> want me to just commit it, should I create an issues, ... ?
[16:44:45 EDT(-0400)] <awills> on that subject – i don't think the effort on CRN is to the point where we have to have an issues ticket associated w/ every commit. I use issues to support a public dialog. If there's no need for a dialog, i don't think it's critical. a descriptive commit comment is great
[16:44:51 EDT(-0400)] <awills> and please go ahead
[16:45:00 EDT(-0400)] <EricDalquist> sounds good
[16:45:11 EDT(-0400)] <EricDalquist> and is it ok if I do a commit for the CacheTask too?
[16:45:36 EDT(-0400)] <awills> what does it do?
[16:45:55 EDT(-0400)] <holdorph> ticket numbers, really assist, weeks, months, years down the road, to figure out why a change was made
[16:46:12 EDT(-0400)] <holdorph> as soon as you allow a change to be made without one, you lose that piece of information history
[16:48:07 EDT(-0400)] <awills> but a peice of software can go through many changes as it's envisioned, before it's released, that are more a process of finding itself... not changing its behavior
[16:49:01 EDT(-0400)] <awills> on that note, i wish googlecode issues tracker was more featuresome
[16:50:15 EDT(-0400)] * holdorph disagrees, but it's not a uportal project, so no use arguing the point here.
[16:56:37 EDT(-0400)] <EricDalquist> yay ... ok dlm:positionSet and dlm:position are exporting correctly ....
[16:57:26 EDT(-0400)] <awills> great
[17:02:15 EDT(-0400)] <EricDalquist> and dlm:origin works now too
[17:08:37 EDT(-0400)] <EricDalquist> awills: do Attributes. values get copied into scope for jexl/groovy?
[17:22:17 EDT(-0400)] <EricDalquist> fyi: http://uportal.pastebin.com/d53486973
[17:22:24 EDT(-0400)] <EricDalquist> that is what an exported layout looks like from my uw
[17:22:55 EDT(-0400)] <EricDalquist> the dlm-hidden stuff is a uw specific extension, but the dlm:origin parameter and the dlm-position stuff are parts of the base framework
[17:25:08 EDT(-0400)] <awills> yes they get coppied... though jexl and groovy are different in how they do it
[17:25:29 EDT(-0400)] <EricDalquist> yeah I figured it out
[17:25:31 EDT(-0400)] <awills> the jexl support is very old, and based on jexl apis directly... the groovy is jsr-223
[17:25:42 EDT(-0400)] <EricDalquist> problem was I hadn't updated the attribute helper for CACHE or CACHE_MODEL
[17:25:57 EDT(-0400)] <awills> ah yeah i meant to look at that specifically
[17:26:26 EDT(-0400)] <EricDalquist> I'm adding lines like:
[17:26:26 EDT(-0400)]
[17:26:30 EDT(-0400)] <EricDalquist> to the end of each type
[17:26:37 EDT(-0400)] <awills> this has been an interesting week... i'm looking forward to the break to catch up and set myself up to cut 1.0.0 GA
[17:26:37 EDT(-0400)] <EricDalquist> just so I have some idea of how much space is being used
[17:27:08 EDT(-0400)] <awills> ah, ok
[17:27:39 EDT(-0400)] <awills> you could consider doing that instead w/ <log> perhaps... just a thought
[17:27:45 EDT(-0400)] <EricDalquist> yeah
[17:28:04 EDT(-0400)] <EricDalquist> oh I also snuck in a change to LogTask accidentally
[17:28:08 EDT(-0400)] <EricDalquist> well
[17:28:10 EDT(-0400)] <EricDalquist> I meant to bring it up
[17:28:17 EDT(-0400)] <EricDalquist> then I forgot I had it there and comitted it
[17:28:34 EDT(-0400)] <EricDalquist> it checks that the log level is enabled before evaluating the message phrase
[17:29:06 EDT(-0400)] <awills> haha... it's all good... not understanding every line of every commit in CRN before it goes in is a problem I'm glad to have
[17:29:18 EDT(-0400)] <awills> it's a step in the right direction
[17:30:32 EDT(-0400)] <awills> i expect you're very interested in seeing your work in there quickly b//c you plan to point the uP pom at 1.0.0 when it's available
[17:30:46 EDT(-0400)] <EricDalquist> honestly not really
[17:30:56 EDT(-0400)] <EricDalquist> just because this is all local work
[17:31:02 EDT(-0400)] <EricDalquist> so I'm not as tied to version numbers
[17:31:16 EDT(-0400)] <EricDalquist> I just hope I'm nearing the end of tweaking these scripts
[17:31:20 EDT(-0400)] <EricDalquist> I have other stuff I need to work on 
[17:31:23 EDT(-0400)] <awills> oh? you're not going to update the imp/exp in uP w/ your changes?
[17:31:29 EDT(-0400)] <EricDalquist> oh I will
[17:31:40 EDT(-0400)] <EricDalquist> but locally I can just as easily use M9-SNAPSHOT
[17:31:49 EDT(-0400)] <EricDalquist> we have a local maven repo I can stick it in
[17:32:19 EDT(-0400)] <awills> even if your .crn files are custom, you'll get most of the performance boost just by pulling a jar that includes all your cache work
[17:32:28 EDT(-0400)] <EricDalquist> of coruse
[17:32:52 EDT(-0400)] <EricDalquist> all I'm doing right now is copying the latest M9 build into our uPortal2 cvs project
[17:33:13 EDT(-0400)] <EricDalquist> eventually when we get this actually tested I'm planning on releasing a 2.X export bundle
[17:33:28 EDT(-0400)] <EricDalquist> kind of a 'drop this into your 2.x install and run this command'
[17:33:35 EDT(-0400)] <awills> ah gotcha
[17:33:49 EDT(-0400)] <EricDalquist> and then make sure that the import scripts in 3.x are update to date to handle the data
[17:38:07 EDT(-0400)] <EricDalquist> hrm ... interesting affect here that I didn't think about
[17:38:13 EDT(-0400)] <EricDalquist> so export.crn calls itself
[17:38:28 EDT(-0400)]
[17:38:54 EDT(-0400)] <EricDalquist> that cache is only scoped to the inner most call for the run of the tasks
[17:39:40 EDT(-0400)]
[17:40:24 EDT(-0400)] <awills> b/c of the concurrency?
[17:40:50 EDT(-0400)] <EricDalquist> those are some of the root elements of export.crn
[17:40:59 EDT(-0400)] <EricDalquist> and since it calls itself
[17:41:05 EDT(-0400)] <awills> ah, the 2 statements are related
[17:41:09 EDT(-0400)] <EricDalquist> they get run again for each recusive call on the stack
[17:41:20 EDT(-0400)] <EricDalquist> and only the bottom most attributes get used
[17:42:52 EDT(-0400)] <awills> maybe it's time to move some of the export operations to another file like 'export-dispatch.crn' or somesuch... setup in the orig file, do recursive stuff in the dispatch file
[17:43:14 EDT(-0400)] <EricDalquist> yeah
[17:43:21 EDT(-0400)] <EricDalquist> that's what I'm going to do
[17:43:26 EDT(-0400)] <EricDalquist> the culprit is the 'all' option
[17:43:35 EDT(-0400)] <EricDalquist> so that logic just needs to be moved into a top level file
[17:49:28 EDT(-0400)] <EricDalquist> eventually I also need to figure out how to detect and not export layouts that just have a header and footer
[17:50:55 EDT(-0400)] <awills> haha
[17:51:21 EDT(-0400)] <awills> that rings a bell... i think there's something similar to that in there
[17:51:37 EDT(-0400)] <EricDalquist> from my testing it looks like there was some change between 2.5 and 3.0 so that now the LM is smart enough to not create that base skeleton until a user actually customizes their layout
[17:51:45 EDT(-0400)] <EricDalquist> but in 2.5 ... we have a .layout for EVERY user
[17:52:00 EDT(-0400)] <EricDalquist> even if most of them are only header/footer
[17:55:41 EDT(-0400)] <EricDalquist> trying to see the best way to do it without having to do another db query
[17:55:52 EDT(-0400)] <EricDalquist> and detect it as early as possible to avoid as much processing as I can ...
[17:58:46 EDT(-0400)] <EricDalquist> would there be an easy way for me to count the number of nodes in the DOM that the script is building as its running?
[18:00:04 EDT(-0400)] <awills> i'm sure... just a sec
[18:00:20 EDT(-0400)]
[18:01:44 EDT(-0400)] <awills> yeah, probably something like that would work... but that would be pretty much when the document is ready to write
[18:01:55 EDT(-0400)] <EricDalquist> yeah
[18:01:59 EDT(-0400)] <EricDalquist> this is in export-layout
[18:02:12 EDT(-0400)] <EricDalquist> it runs: SELECT * FROM up_layout_struct WHERE user_id = ? AND layout_id = 1 AND chan_id IS NULL
[18:02:28 EDT(-0400)] <EricDalquist> and I figure if I count the number of results I can figure out if it is an empty layout or not
[18:02:55 EDT(-0400)] <EricDalquist> but the best was I can think of to do that is to let the subtasks for the query run and build the dom then count entries
[18:03:16 EDT(-0400)] <awills> probably, though you'd probably need a # that's meaningful for your institution
[18:03:49 EDT(-0400)] <EricDalquist> yeah
[18:03:54 EDT(-0400)]
[18:03:56 EDT(-0400)] <EricDalquist> this would be a UW specific thing in the export
[18:04:27 EDT(-0400)] <EricDalquist> well we have things other than folders now that there is dlm support
[18:05:33 EDT(-0400)]
[18:06:11 EDT(-0400)] <awills> (that's an approximation btw, need to confirm the details)
[18:06:12 EDT(-0400)] <EricDalquist> ah yeah
[18:09:25 EDT(-0400)]
[18:09:34 EDT(-0400)] <EricDalquist> if that is 0 its an empty layout
[18:09:42 EDT(-0400)] <awills> sounds good
[18:10:52 EDT(-0400)] <EricDalquist> went from exporting 700 .layout files to 100
[18:11:09 EDT(-0400)] <EricDalquist> and we can skip several queries and all of the XSL and stuff for those users then too
[18:11:36 EDT(-0400)] <awills> that's something... even if you're still doing most of the processing on export, you'll really see a difference on import
[18:12:41 EDT(-0400)] <EricDalquist> well and it should be a big help on export since the db is our bottleneck
[18:12:53 EDT(-0400)] <EricDalquist> and we're skiping all the join queries to grab the extra layout info
[18:12:56 EDT(-0400)] <EricDalquist> hrm
[18:13:00 EDT(-0400)] <EricDalquist> crap
[18:13:08 EDT(-0400)] <EricDalquist> I also need to check if they have portlet preferences
[18:13:26 EDT(-0400)] <awills> yes, that's true
[18:15:06 EDT(-0400)] <awills> you have been at this effort tenaciously for days... i'm blown away by the difference you've made... but I do hope you occasionally take a deep breath and ask yourself if the next improvement is worth the extra work & complexity
[18:15:16 EDT(-0400)] <awills> you're the best one to judge though
[18:15:37 EDT(-0400)] <EricDalquist> well I'm not as much on performance now
[18:15:41 EDT(-0400)] <EricDalquist> as manageability
[18:15:57 EDT(-0400)] <EricDalquist> we need to get the export dataset down to as reasonable a size as possible
[18:16:02 EDT(-0400)] <awills> +1 for manageability
[18:17:27 EDT(-0400)] <EricDalquist> hrm one use-case I know we won't hit but we should test for 3.0
[18:17:39 EDT(-0400)] <EricDalquist> what happens if you have no layout customizations so nothing in layout_struct
[18:17:46 EDT(-0400)] <EricDalquist> but portlet entity prefs
[18:17:51 EDT(-0400)] <EricDalquist> does the current export detect that
[18:18:11 EDT(-0400)] <EricDalquist> I think right now you'd loose those
[18:18:13 EDT(-0400)] <awills> possibly
[18:18:20 EDT(-0400)] <EricDalquist> oh well
[18:18:22 EDT(-0400)] <EricDalquist> my ride is here
[18:18:25 EDT(-0400)] <EricDalquist> off to a nice long weekendf
[18:18:33 EDT(-0400)] <awills> yeah have fun
[18:18:39 EDT(-0400)] <EricDalquist> have fun relaxing everyone!
[20:17:37 EDT(-0400)] * awills (n=awills@wsip-98-174-242-39.ph.ph.cox.net) has left ##uportal
[21:03:06 EDT(-0400)] * Sememmon (n=Sememmon@ip70-190-242-64.ph.ph.cox.net) has joined ##uportal
[21:03:06 EDT(-0400)] * Tuomaz (n=fredrik@kaffe.umdc.umu.se) has joined ##uportal
[21:03:06 EDT(-0400)] * higpetter (n=petter@brunte.HIG.SE) has joined ##uportal