uPortal IRC Logs-2012-04-09

Owned by Apereo Infrastructure
Last updated: Apr 09, 2012
5 min read

[12:54:13 CDT(-0500)] <b-sure> Hello uPortal Devs: I've got an error in our dev environment for a user http://pastebin.com/raw.php?i=WuEDACjB do you know if this is a sign of missing data or maybe a corrupt layout?

[12:58:59 CDT(-0500)] <EricDalquist> weird

[12:59:00 CDT(-0500)] <EricDalquist> yeah

[12:59:03 CDT(-0500)] <EricDalquist> must be an invalid layout

[12:59:17 CDT(-0500)] <b-sure> should I try to reimport his layout file?

[12:59:23 CDT(-0500)] <EricDalquist> yeah

[12:59:31 CDT(-0500)] <EricDalquist> see if you get any weird warnings/errors

[12:59:34 CDT(-0500)] <b-sure> ok. I'll start w/ that. Thanks EricDalquist

[12:59:36 CDT(-0500)] <b-sure> ok

[13:37:21 CDT(-0500)] <b-sure> ok. reimporting the users layout file fixed the issue for us.

[13:38:49 CDT(-0500)] <b-sure> EricDalquist. I think you mentioned last week that you have made some adjustments to the 3.2-patches export routine. Are those available in the github repo? I'd like to try them and then do an import of the data to 4.x

[13:45:51 CDT(-0500)] <athena> EricDalquist: i'm finally back to taking a look at the login aggregation stats reporting

[13:46:05 CDT(-0500)] <athena> it looks like for days w/ no logins it doesn't return a date dimension

[13:46:09 CDT(-0500)] <athena> is that the expected behavior?

[14:01:30 CDT(-0500)] <b-sure> Hi EricDalquist, I've got another layout related question: http://pastebin.com/raw.php?i=A7JJyQxQ looks like were down to our last type of import error from 3.2-patches to 4.x. Do you know if this type of issue is addressed in your recent mods to the import process?

[16:02:52 CDT(-0500)] <EricDalquist> athena / b-sure I'm back

[16:02:54 CDT(-0500)] <EricDalquist> athena: yes

[16:03:14 CDT(-0500)] <EricDalquist> though you could probably add some logic to the aggregator to change that behavior and populate empty rowsa

[16:08:51 CDT(-0500)] <EricDalquist> 0 5 * * 2

[16:08:55 CDT(-0500)] <EricDalquist> does that look right?

[16:09:05 CDT(-0500)] <EricDalquist> 5am every Tuesday

[16:21:10 CDT(-0500)] <athena> yes, i think that makes sense

[16:21:15 CDT(-0500)] <athena> so yes, i think we do want empty rows

[16:21:17 CDT(-0500)] <EricDalquist> oops

[16:21:19 CDT(-0500)] <athena> otherwise the graphs don't really make sense

[16:21:21 CDT(-0500)] <EricDalquist> wrong chat window for that cron stuff

[16:21:25 CDT(-0500)] <athena> lol

[16:22:10 CDT(-0500)] <athena> so either we need the aggregator to drop down 0-value rows, or we need to have special handling when we do graphing

[16:22:43 CDT(-0500)] <EricDalquist> well from a DB space saving PVO

[16:22:45 CDT(-0500)] <EricDalquist> POV*

[16:22:50 CDT(-0500)] <EricDalquist> I like missing rows (smile)

[16:23:04 CDT(-0500)] <EricDalquist> when the DAO returns the result it could run through the List

[16:23:07 CDT(-0500)] <EricDalquist> and fill in gaps

[16:24:06 CDT(-0500)] <athena> yeah

[16:27:47 CDT(-0500)] <EricDalquist> I know it may not seem like much but over the years having sparse data will really help

[16:28:21 CDT(-0500)] <athena> yeah

[16:28:23 CDT(-0500)] <athena> that makes sense

[16:31:05 CDT(-0500)] <EricDalquist> yay comcast (smile)

[16:37:27 CDT(-0500)] <athena> no kidding.

[16:37:30 CDT(-0500)] * athena screams

[16:38:35 CDT(-0500)] <athena> clearly they're having a bad day - i'm on the edge of migrating to a coffee shop or UW campus or something

[16:38:35 CDT(-0500)] <athena> so w/ the DAO

[16:38:35 CDT(-0500)] <athena> i guess we might need a new LoginAggregation implementation?

[16:38:35 CDT(-0500)] <athena> so that we can construct new aggregations that aren't JPA-persisted?

[16:40:03 CDT(-0500)] <EricDalquist> sure ... I'd probably create a EmptyLoginAggregation implements LoginAggregation

[16:40:11 CDT(-0500)] <EricDalquist> and have a singleton INSTANCE of it

[16:40:22 CDT(-0500)] <athena> sounds terrific - along the lines of what i was thinking

[16:40:36 CDT(-0500)] <athena> like the suggestion about the singleton

[16:40:36 CDT(-0500)] <EricDalquist> then add some logic in the display code

[16:40:53 CDT(-0500)] <EricDalquist> that would run through the list and fill in blanks

[16:40:56 CDT(-0500)] <athena> yeah

[16:41:01 CDT(-0500)] <EricDalquist> if I was really feeling enterprising ...

[16:41:03 CDT(-0500)] <athena> do we want that in the display code or the dao?

[16:41:14 CDT(-0500)] <EricDalquist> display code

[16:41:16 CDT(-0500)] <athena> ok

[16:41:20 CDT(-0500)] <EricDalquist> let the DAO represent what is actually in the data

[16:41:28 CDT(-0500)] <athena> well, in that case we don't actually need an aggregation at all

[16:41:35 CDT(-0500)] <athena> since the display models things as a google DataTable

[16:41:44 CDT(-0500)] <EricDalquist> Ah

[16:41:51 CDT(-0500)] <EricDalquist> well the reason I was thinking an aggreagation is

[16:41:59 CDT(-0500)] <EricDalquist> I'd consider adding an abstract method to AggregationInterval

[16:42:03 CDT(-0500)] <athena> i worry a bit about having display logic that's too complex just because it might make it harder for adopters to add new reports

[16:42:07 CDT(-0500)] <EricDalquist> which each interval implements

[16:42:44 CDT(-0500)] <EricDalquist> public abstract <T extends BaseAggregation> List<T> fillInBlanks(List<T> data, T blank);

[16:42:56 CDT(-0500)] <EricDalquist> that lets you re-use this logic for other aggregations down the road

[16:43:10 CDT(-0500)] <EricDalquist> since the "fill in the blank" logic is going to be interval specific

[16:43:11 CDT(-0500)] <EricDalquist> hrm

[16:43:12 CDT(-0500)] <EricDalquist> actuall

[16:43:16 CDT(-0500)] <EricDalquist> damn terms/quarters

[16:43:28 CDT(-0500)] <EricDalquist> because of those

[16:43:40 CDT(-0500)] <EricDalquist> you should add "public <T extends BaseAggregation> List<T> fillInBlanks(List<T> data, T blank)" to AggregationIntervalHelper

[16:43:50 CDT(-0500)] <EricDalquist> and implement it in AggregationIntervalHelperImpl

[16:44:05 CDT(-0500)] <EricDalquist> since you'll need DB lookups to figure if there are missing terms/quarters

[16:44:14 CDT(-0500)] <EricDalquist> and a singleton won't work

[16:44:20 CDT(-0500)] <EricDalquist> since the date/time fields need to be correct

[16:44:21 CDT(-0500)] <EricDalquist> damnit

[16:45:04 CDT(-0500)] <EricDalquist> public <T extends BaseAggregation> List<T> fillInBlanks(List<T> data, Function<DateTime, T> missingDataCreator);

[16:45:05 CDT(-0500)] <EricDalquist> there

[16:45:12 CDT(-0500)] <EricDalquist> where Function is a com.google.common.base.Function

[16:45:23 CDT(-0500)] <athena> ok

[16:45:31 CDT(-0500)] <EricDalquist> then that can call missingDataCreator.apply(missingDateTime)

[16:45:48 CDT(-0500)] <athena> so basically we're adding a utility method to the interval helper that will help fill in missing holes and return a new list of aggregations

[16:45:53 CDT(-0500)] <EricDalquist> yes

[16:46:10 CDT(-0500)] <athena> and we'll need logic specific to each aggregation type

[16:46:15 CDT(-0500)] <EricDalquist> just because the "find the hole" logic isn't straight forward for all intervals

[16:46:20 CDT(-0500)] <athena> so that it can create aggregation-specific empty objects

[16:47:00 CDT(-0500)] <EricDalquist> yeah, which is what that Function should be doing

[16:47:02 CDT(-0500)] <EricDalquist> grr

[16:47:03 CDT(-0500)] <EricDalquist> ok

[16:47:03 CDT(-0500)] <EricDalquist> public <T extends BaseAggregation> List<T> fillInBlanks(List<T> data, Function<Tuple<DateDimension, TimeDimension>, T> missingDataCreator);

[16:47:06 CDT(-0500)] <EricDalquist> there

[16:47:17 CDT(-0500)] <EricDalquist> the fillInTheBlank function will have to figure out the date and time dimensions

[16:47:19 CDT(-0500)] <EricDalquist> and provide those

[16:47:29 CDT(-0500)] <EricDalquist> just a DateTime wouldn't be enough info for the function

[16:47:31 CDT(-0500)] <EricDalquist> blah

[16:47:59 CDT(-0500)] <athena> yeah

[16:48:01 CDT(-0500)] <EricDalquist> so you'd get the aggregation from the dao

[16:48:01 CDT(-0500)] <athena> not trivial

[16:48:07 CDT(-0500)] <EricDalquist> run the list through fillInBlanks

[16:48:10 CDT(-0500)] <athena> but if we can at least keep this out of the controller that'll help

[16:48:14 CDT(-0500)] <EricDalquist> yes

[16:48:22 CDT(-0500)] <EricDalquist> keeping it in one method will be good

[16:48:30 CDT(-0500)] <athena> i finally got something reasonable working displaying the graphs

[16:48:35 CDT(-0500)] <EricDalquist> then use that list to build the DataTable

[16:48:38 CDT(-0500)] <EricDalquist> yay!

[16:48:46 CDT(-0500)] <athena> managed to use google's logic to serialize the DataTable to a JSON fragment without it taking over the whole response

[16:48:47 CDT(-0500)] <athena> yeah

[16:48:52 CDT(-0500)] <EricDalquist> "sometime soon" I'll be working on more aggregators

[16:48:56 CDT(-0500)] <EricDalquist> oh nice

[16:48:56 CDT(-0500)] <athena> so we shoudl be able to include multiple tables in a response now, etc.

[16:49:08 CDT(-0500)] <athena> once i get all this working i'll check it in and then we can have a real example to work from

[16:49:11 CDT(-0500)] <EricDalquist> yay for reverse engineering (smile)

[16:49:12 CDT(-0500)] <athena> just keep finding little things

[16:49:13 CDT(-0500)] <EricDalquist> wonderful

[16:49:13 CDT(-0500)] <athena> lol

[16:49:13 CDT(-0500)] <athena> yeah

[16:49:17 CDT(-0500)] <EricDalquist> yeah

[16:49:20 CDT(-0500)] <athena> yay for being able to look at source code online

[16:49:22 CDT(-0500)] <EricDalquist> I think we found an aggregator bug

[16:49:28 CDT(-0500)] <EricDalquist> so have to fix that soon

[16:49:33 CDT(-0500)] <EricDalquist> but time for me to head home

[16:49:37 CDT(-0500)] <EricDalquist> have a good evening (smile)

[16:50:06 CDT(-0500)] <athena> you too!