[10:56:20 CDT(-0500)] <EricDalquist> this timezone stuff is going to drive me insane!
[10:58:40 CDT(-0500)] <athena> noooooo!
[10:58:47 CDT(-0500)] <athena> pull back from the dark side!
[11:16:14 CDT(-0500)] <athena> so our data model for fragment defs allows more than one audience
[11:16:25 CDT(-0500)] <athena> but i don't think that actually works, does it?
[11:16:39 CDT(-0500)] <athena> i suspect that may be an artifact of shoehorning those into the existing data structures
[11:22:30 CDT(-0500)] <EricDalquist> I'm not sure
[11:22:40 CDT(-0500)] <EricDalquist> I've purposely not really looked at it
[11:22:55 CDT(-0500)] <EricDalquist> as if I do I feel an overwhelming urge to refactor it all into sanity
[11:23:06 CDT(-0500)] <EricDalquist> but yes
[11:23:13 CDT(-0500)] <EricDalquist> I believe you can target multiple audiences
[11:47:29 CDT(-0500)] <EricDalquist> @#$!@#$
[11:47:51 CDT(-0500)] <EricDalquist> The JDBC specification does not define any details with regards to time zone
[11:47:55 CDT(-0500)] <EricDalquist> yay
[11:48:13 CDT(-0500)] <athena> ummmmm
[11:48:14 CDT(-0500)] <athena> wow.
[11:48:25 CDT(-0500)] <athena> hey that tz stuff isn't important, lets ignore it 
[11:48:29 CDT(-0500)] <EricDalquist> seriously
[11:48:33 CDT(-0500)] <athena> crazy
[11:48:35 CDT(-0500)] <EricDalquist> I'm going to start throwing things
[11:48:55 CDT(-0500)] <athena> hands over more objects to throw
[13:14:31 CDT(-0500)] <EricDalquist> hrm
[13:14:37 CDT(-0500)] <EricDalquist> maybe this is just a hsqldb bug
[13:14:40 CDT(-0500)] <EricDalquist> time to test oracle
[13:14:48 CDT(-0500)] <EricDalquist> I've at least isolated the problem
[13:15:13 CDT(-0500)] <EricDalquist> the jodatime mapping coverts the LocalDate of 2012-10-15 into a sql Date object
[13:15:20 CDT(-0500)] <EricDalquist> which ends up looking like "2012-10-14T19:00:00.000-0500"
[13:15:29 CDT(-0500)] <EricDalquist> the problem is hsql seems to store that in the local time zone
[13:15:37 CDT(-0500)] <EricDalquist> so it stores "2012-10-14"
[13:15:52 CDT(-0500)] <EricDalquist> and on the way back out of the DB the sql Date object is set to "2012-10-14T00:00:00.000-0500"
[13:19:08 CDT(-0500)] <EricDalquist> I wonder if this is a hsqldb bug even
[13:19:31 CDT(-0500)] <EricDalquist> since java.sql.Date says it is supposed to track seconds since GMT epoch
[13:19:39 CDT(-0500)] <EricDalquist> which is what the value that is passed in was
[13:54:52 CDT(-0500)] <athena> interesting
[14:01:45 CDT(-0500)] <EricDalquist> angering more like it 
[14:28:47 CDT(-0500)] <athena> well
[14:28:55 CDT(-0500)] <athena> an hsqldb bugt sounds better than a jdbc bug?
[14:28:59 CDT(-0500)] <EricDalquist> yeah
[14:29:16 CDT(-0500)] <EricDalquist> I think I might have a full scale test involving plan JDBC access to figure this out :/
[14:29:30 CDT(-0500)] <EricDalquist> my fear is that the mapping from the joda LocalDate -> JDBC is wrong
[14:29:36 CDT(-0500)] <athena> ergh :/
[14:29:51 CDT(-0500)] <EricDalquist> and we're going to have to figure out how to modify a bunch of already processed stats data
[14:30:42 CDT(-0500)] <athena> eep
[14:30:45 CDT(-0500)] <EricDalquist> yeah
[15:08:13 CDT(-0500)] <EricDalquist> yup ... its official
[15:08:18 CDT(-0500)] <EricDalquist> the JDBC Date type sucks
[15:08:48 CDT(-0500)] <EricDalquist> and I'm honestly not sure what the fix is
[15:08:56 CDT(-0500)] <EricDalquist> since TZ stuff is so freaking nebulous
[15:09:45 CDT(-0500)] <EricDalquist> https://gist.github.com/3901664
[15:09:47 CDT(-0500)] <EricDalquist> that fails
[15:09:50 CDT(-0500)] <EricDalquist> with both hsql and oracle
[15:10:27 CDT(-0500)] <EricDalquist> in both cases my LocalDate(2012-10-15) gets coverted to a JDBC Date object with an internal value of "2012-10-14T19:00:00.000-0500"
[15:10:37 CDT(-0500)] <EricDalquist> and the DB only stores the y/m/d biy
[15:10:38 CDT(-0500)] <EricDalquist> bit
[15:10:49 CDT(-0500)] <EricDalquist> so it comes back out as ""2012-10-14T00:00:00.000-0500""
[15:26:10 CDT(-0500)] <EricDalquist> UHG
[15:26:15 CDT(-0500)] <EricDalquist> so I think I figured out a fix ...
[15:27:07 CDT(-0500)] <EricDalquist> so it looks like the usertype library should be setting the databaseZone to the current JVM TZ
[15:27:26 CDT(-0500)] <EricDalquist> instead of UTC
[15:27:34 CDT(-0500)] <EricDalquist> the problem is when I make that change
[15:28:03 CDT(-0500)] <EricDalquist> aggregation is going to jump ahead by 1 day if you have a negative utc offset
[15:28:10 CDT(-0500)] <EricDalquist> or behind by 1 day if you have a positive utc offset
[15:28:13 CDT(-0500)] <EricDalquist> which is bad
[15:28:39 CDT(-0500)] <athena> oh 
[15:29:17 CDT(-0500)] <EricDalquist> grrr
[15:29:27 CDT(-0500)] * EricDalquist really hates timezones
[15:29:33 CDT(-0500)] <athena> no kidding.
[15:29:55 CDT(-0500)] <EricDalquist> time to figure out a way to fix this ...
[15:30:13 CDT(-0500)] <EricDalquist> the big problem is the people with a positive TZ offset
[15:30:22 CDT(-0500)] <EricDalquist> for the negative people they get a hole in their stats data
[15:30:25 CDT(-0500)] <EricDalquist> which isn't horrible
[15:30:32 CDT(-0500)] <EricDalquist> but for positive TZ offsets
[15:30:33 CDT(-0500)] <athena> yeah
[15:30:37 CDT(-0500)] <EricDalquist> it will crash the aggregation
[15:30:38 CDT(-0500)] <athena> that's not so good
[15:30:52 CDT(-0500)] <EricDalquist> since data will already exist and be in a "closed" state
[15:31:02 CDT(-0500)] <EricDalquist> so that means to fix this
[15:31:12 CDT(-0500)] <EricDalquist> 1. disable stats aggregation on ALL machines in the environment
[15:31:30 CDT(-0500)] <EricDalquist> 2. run program to "fix" dates (not sure how that will work yet)
[15:31:38 CDT(-0500)] <EricDalquist> 3. roll out new portal code
[15:31:51 CDT(-0500)] <EricDalquist> 3 would enable aggregation again
[15:31:53 CDT(-0500)] <EricDalquist> grr
[15:32:13 CDT(-0500)] <athena> yeah :/
[15:37:29 CDT(-0500)] <EricDalquist> I have a feature I realized I forgot in the works
[15:37:48 CDT(-0500)] <EricDalquist> that essentially stops aggreattion from running where the code version is less than the db version
[15:38:07 CDT(-0500)] <EricDalquist> so if you upgrade one server of a cluster from 4.0.7 to 4.0.8
[15:38:11 CDT(-0500)] <EricDalquist> and that includes running db-update
[15:38:24 CDT(-0500)] <EricDalquist> only server(s) running 4.0.8 will do even aggreation
[15:38:37 CDT(-0500)] <athena> yeah
[15:38:38 CDT(-0500)] <EricDalquist> if that was in place now I think I could mostly automate this process
[15:38:43 CDT(-0500)] <EricDalquist> but it isn't
[15:38:47 CDT(-0500)] <athena> oh
[15:38:53 CDT(-0500)] <EricDalquist> the db versioning is there
[15:38:59 CDT(-0500)] <EricDalquist> but I forgot the check in the aggregator
[15:39:41 CDT(-0500)] <athena> ah
[15:39:54 CDT(-0500)] <EricDalquist> well ... I'll do a bunch of testing here
[15:40:00 CDT(-0500)] <EricDalquist> and see what I can figure out for a fix
[15:40:02 CDT(-0500)] <EricDalquist>
[16:30:51 CDT(-0500)] <EricDalquist> yup ... fixed the bug
[16:31:04 CDT(-0500)] <EricDalquist> and now all my event related dates jumped back in time by 5 hours
[16:31:06 CDT(-0500)] <EricDalquist>
[16:42:41 CDT(-0500)] <EricDalquist> uhg ... 8 columns across 5 tables affected by this TZ issue
[16:42:47 CDT(-0500)] <EricDalquist> this is not good at all
[16:43:20 CDT(-0500)] <EricDalquist> hrm
[16:43:33 CDT(-0500)] <EricDalquist> though setting the TZ to "jvm" isn't great either
[16:43:46 CDT(-0500)] <EricDalquist> I need to test what happens when the server crosses a DST boundary
[16:47:21 CDT(-0500)] <EricDalquist> so really it seems like storing stuff in UTC would be the best bet
[16:47:28 CDT(-0500)] <EricDalquist> the problem is this LocalDate ....
[16:47:39 CDT(-0500)] <EricDalquist> and its interaction with Date
[16:50:36 CDT(-0500)] <athena> yikes
[16:50:44 CDT(-0500)] <athena> that all sounds kind of concerning
[16:50:50 CDT(-0500)] <EricDalquist> yeah
[16:50:51 CDT(-0500)] <EricDalquist> it is
[16:50:54 CDT(-0500)] <EricDalquist> oh
[16:50:55 CDT(-0500)] <EricDalquist> ok
[16:50:56 CDT(-0500)] <EricDalquist> just tested
[16:51:02 CDT(-0500)] <EricDalquist> it is handling DST correctly
[16:51:08 CDT(-0500)] <EricDalquist> when using the jvm TZ
[16:51:10 CDT(-0500)] <EricDalquist> I believe
[16:51:11 CDT(-0500)] <EricDalquist> still
[16:51:16 CDT(-0500)] <EricDalquist> more testing is going to be needed
[16:51:25 CDT(-0500)] <EricDalquist> and probably a fairly complex java app will be needed to "fix" this
[16:51:28 CDT(-0500)] <EricDalquist>