[12:54:13 CDT(-0500)] <b-sure> Hello uPortal Devs: I've got an error in our dev environment for a user http://pastebin.com/raw.php?i=WuEDACjB do you know if this is a sign of missing data or maybe a corrupt layout?
[12:58:59 CDT(-0500)] <EricDalquist> weird
[12:59:00 CDT(-0500)] <EricDalquist> yeah
[12:59:03 CDT(-0500)] <EricDalquist> must be an invalid layout
[12:59:17 CDT(-0500)] <b-sure> should I try to reimport his layout file?
[12:59:23 CDT(-0500)] <EricDalquist> yeah
[12:59:31 CDT(-0500)] <EricDalquist> see if you get any weird warnings/errors
[12:59:34 CDT(-0500)] <b-sure> ok. I'll start w/ that. Thanks EricDalquist
[12:59:36 CDT(-0500)] <b-sure> ok
[13:37:21 CDT(-0500)] <b-sure> ok. reimporting the users layout file fixed the issue for us.
[13:38:49 CDT(-0500)] <b-sure> EricDalquist. I think you mentioned last week that you have made some adjustments to the 3.2-patches export routine. Are those available in the github repo? I'd like to try them and then do an import of the data to 4.x
[13:45:51 CDT(-0500)] <athena> EricDalquist: i'm finally back to taking a look at the login aggregation stats reporting
[13:46:05 CDT(-0500)] <athena> it looks like for days w/ no logins it doesn't return a date dimension
[13:46:09 CDT(-0500)] <athena> is that the expected behavior?
[14:01:30 CDT(-0500)] <b-sure> Hi EricDalquist, I've got another layout related question: http://pastebin.com/raw.php?i=A7JJyQxQ looks like were down to our last type of import error from 3.2-patches to 4.x. Do you know if this type of issue is addressed in your recent mods to the import process?
[16:02:52 CDT(-0500)] <EricDalquist> athena / b-sure I'm back
[16:02:54 CDT(-0500)] <EricDalquist> athena: yes
[16:03:14 CDT(-0500)] <EricDalquist> though you could probably add some logic to the aggregator to change that behavior and populate empty rowsa
[16:08:51 CDT(-0500)] <EricDalquist> 0 5 * * 2
[16:08:55 CDT(-0500)] <EricDalquist> does that look right?
[16:09:05 CDT(-0500)] <EricDalquist> 5am every Tuesday
[16:21:10 CDT(-0500)] <athena> yes, i think that makes sense
[16:21:15 CDT(-0500)] <athena> so yes, i think we do want empty rows
[16:21:17 CDT(-0500)] <EricDalquist> oops
[16:21:19 CDT(-0500)] <athena> otherwise the graphs don't really make sense
[16:21:21 CDT(-0500)] <EricDalquist> wrong chat window for that cron stuff
[16:21:25 CDT(-0500)] <athena> lol
[16:22:10 CDT(-0500)] <athena> so either we need the aggregator to drop down 0-value rows, or we need to have special handling when we do graphing
[16:22:43 CDT(-0500)] <EricDalquist> well from a DB space saving PVO
[16:22:45 CDT(-0500)] <EricDalquist> POV*
[16:22:50 CDT(-0500)] <EricDalquist> I like missing rows 
[16:23:04 CDT(-0500)] <EricDalquist> when the DAO returns the result it could run through the List
[16:23:07 CDT(-0500)] <EricDalquist> and fill in gaps
[16:24:06 CDT(-0500)] <athena> yeah
[16:27:47 CDT(-0500)] <EricDalquist> I know it may not seem like much but over the years having sparse data will really help
[16:28:21 CDT(-0500)] <athena> yeah
[16:28:23 CDT(-0500)] <athena> that makes sense
[16:31:05 CDT(-0500)] <EricDalquist> yay comcast
[16:37:27 CDT(-0500)] <athena> no kidding.
[16:37:30 CDT(-0500)] * athena screams
[16:38:35 CDT(-0500)] <athena> clearly they're having a bad day - i'm on the edge of migrating to a coffee shop or UW campus or something
[16:38:35 CDT(-0500)] <athena> so w/ the DAO
[16:38:35 CDT(-0500)] <athena> i guess we might need a new LoginAggregation implementation?
[16:38:35 CDT(-0500)] <athena> so that we can construct new aggregations that aren't JPA-persisted?
[16:40:03 CDT(-0500)] <EricDalquist> sure ... I'd probably create a EmptyLoginAggregation implements LoginAggregation
[16:40:11 CDT(-0500)] <EricDalquist> and have a singleton INSTANCE of it
[16:40:22 CDT(-0500)] <athena> sounds terrific - along the lines of what i was thinking
[16:40:36 CDT(-0500)] <athena> like the suggestion about the singleton
[16:40:36 CDT(-0500)] <EricDalquist> then add some logic in the display code
[16:40:53 CDT(-0500)] <EricDalquist> that would run through the list and fill in blanks
[16:40:56 CDT(-0500)] <athena> yeah
[16:41:01 CDT(-0500)] <EricDalquist> if I was really feeling enterprising ...
[16:41:03 CDT(-0500)] <athena> do we want that in the display code or the dao?
[16:41:14 CDT(-0500)] <EricDalquist> display code
[16:41:16 CDT(-0500)] <athena> ok
[16:41:20 CDT(-0500)] <EricDalquist> let the DAO represent what is actually in the data
[16:41:28 CDT(-0500)] <athena> well, in that case we don't actually need an aggregation at all
[16:41:35 CDT(-0500)] <athena> since the display models things as a google DataTable
[16:41:44 CDT(-0500)] <EricDalquist> Ah
[16:41:51 CDT(-0500)] <EricDalquist> well the reason I was thinking an aggreagation is
[16:41:59 CDT(-0500)] <EricDalquist> I'd consider adding an abstract method to AggregationInterval
[16:42:03 CDT(-0500)] <athena> i worry a bit about having display logic that's too complex just because it might make it harder for adopters to add new reports
[16:42:07 CDT(-0500)] <EricDalquist> which each interval implements
[16:42:44 CDT(-0500)] <EricDalquist> public abstract <T extends BaseAggregation> List<T> fillInBlanks(List<T> data, T blank);
[16:42:56 CDT(-0500)] <EricDalquist> that lets you re-use this logic for other aggregations down the road
[16:43:10 CDT(-0500)] <EricDalquist> since the "fill in the blank" logic is going to be interval specific
[16:43:11 CDT(-0500)] <EricDalquist> hrm
[16:43:12 CDT(-0500)] <EricDalquist> actuall
[16:43:16 CDT(-0500)] <EricDalquist> damn terms/quarters
[16:43:28 CDT(-0500)] <EricDalquist> because of those
[16:43:40 CDT(-0500)] <EricDalquist> you should add "public <T extends BaseAggregation> List<T> fillInBlanks(List<T> data, T blank)" to AggregationIntervalHelper
[16:43:50 CDT(-0500)] <EricDalquist> and implement it in AggregationIntervalHelperImpl
[16:44:05 CDT(-0500)] <EricDalquist> since you'll need DB lookups to figure if there are missing terms/quarters
[16:44:14 CDT(-0500)] <EricDalquist> and a singleton won't work
[16:44:20 CDT(-0500)] <EricDalquist> since the date/time fields need to be correct
[16:44:21 CDT(-0500)] <EricDalquist> damnit
[16:45:04 CDT(-0500)] <EricDalquist> public <T extends BaseAggregation> List<T> fillInBlanks(List<T> data, Function<DateTime, T> missingDataCreator);
[16:45:05 CDT(-0500)] <EricDalquist> there
[16:45:12 CDT(-0500)] <EricDalquist> where Function is a com.google.common.base.Function
[16:45:23 CDT(-0500)] <athena> ok
[16:45:31 CDT(-0500)] <EricDalquist> then that can call missingDataCreator.apply(missingDateTime)
[16:45:48 CDT(-0500)] <athena> so basically we're adding a utility method to the interval helper that will help fill in missing holes and return a new list of aggregations
[16:45:53 CDT(-0500)] <EricDalquist> yes
[16:46:10 CDT(-0500)] <athena> and we'll need logic specific to each aggregation type
[16:46:15 CDT(-0500)] <EricDalquist> just because the "find the hole" logic isn't straight forward for all intervals
[16:46:20 CDT(-0500)] <athena> so that it can create aggregation-specific empty objects
[16:47:00 CDT(-0500)] <EricDalquist> yeah, which is what that Function should be doing
[16:47:02 CDT(-0500)] <EricDalquist> grr
[16:47:03 CDT(-0500)] <EricDalquist> ok
[16:47:03 CDT(-0500)] <EricDalquist> public <T extends BaseAggregation> List<T> fillInBlanks(List<T> data, Function<Tuple<DateDimension, TimeDimension>, T> missingDataCreator);
[16:47:06 CDT(-0500)] <EricDalquist> there
[16:47:17 CDT(-0500)] <EricDalquist> the fillInTheBlank function will have to figure out the date and time dimensions
[16:47:19 CDT(-0500)] <EricDalquist> and provide those
[16:47:29 CDT(-0500)] <EricDalquist> just a DateTime wouldn't be enough info for the function
[16:47:31 CDT(-0500)] <EricDalquist> blah
[16:47:59 CDT(-0500)] <athena> yeah
[16:48:01 CDT(-0500)] <EricDalquist> so you'd get the aggregation from the dao
[16:48:01 CDT(-0500)] <athena> not trivial
[16:48:07 CDT(-0500)] <EricDalquist> run the list through fillInBlanks
[16:48:10 CDT(-0500)] <athena> but if we can at least keep this out of the controller that'll help
[16:48:14 CDT(-0500)] <EricDalquist> yes
[16:48:22 CDT(-0500)] <EricDalquist> keeping it in one method will be good
[16:48:30 CDT(-0500)] <athena> i finally got something reasonable working displaying the graphs
[16:48:35 CDT(-0500)] <EricDalquist> then use that list to build the DataTable
[16:48:38 CDT(-0500)] <EricDalquist> yay!
[16:48:46 CDT(-0500)] <athena> managed to use google's logic to serialize the DataTable to a JSON fragment without it taking over the whole response
[16:48:47 CDT(-0500)] <athena> yeah
[16:48:52 CDT(-0500)] <EricDalquist> "sometime soon" I'll be working on more aggregators
[16:48:56 CDT(-0500)] <EricDalquist> oh nice
[16:48:56 CDT(-0500)] <athena> so we shoudl be able to include multiple tables in a response now, etc.