[06:09:35 CDT(-0500)] * athena (~athena@c-76-121-97-221.hsd1.wa.comcast.net) has joined ##uportal
[08:14:55 CDT(-0500)] * colinclark (~colin@bas2-toronto09-1176130958.dsl.bell.ca) has joined ##uportal
[09:41:19 CDT(-0500)] * colinclark (~colin@bas2-toronto09-1176130958.dsl.bell.ca) has joined ##uportal
[09:51:01 CDT(-0500)] * lfuller (~sparhk@wsip-72-215-204-133.ph.ph.cox.net) has joined ##uportal
[09:56:42 CDT(-0500)] * athena (~athena@c-76-121-97-221.hsd1.wa.comcast.net) has joined ##uportal
[09:58:27 CDT(-0500)] * EricDalquist (~dalquist@2607:f388:e:0:221:9bff:fe37:e768) has joined ##uportal
[10:02:04 CDT(-0500)] * bsparks (~bsparks@wsip-72-215-204-133.ph.ph.cox.net) has joined ##uportal
[10:04:14 CDT(-0500)] * bsparks (~bsparks@wsip-72-215-204-133.ph.ph.cox.net) has joined ##uportal
[10:07:08 CDT(-0500)] * EricDalquist just discovered spring bean overriding
[10:11:20 CDT(-0500)] <EricDalquist> also: http://maven.apache.org/plugins/maven-war-plugin/faq.html#attached
[10:23:43 CDT(-0500)] * JoeMoore (89d88117@gateway/web/freenode/ip.137.216.129.23) has joined ##uportal
[10:26:35 CDT(-0500)] * awills (~awills@wsip-72-215-204-133.ph.ph.cox.net) has joined ##uportal
[10:33:02 CDT(-0500)] <athena> nice
[10:45:14 CDT(-0500)] * colinclark (~colin@bas2-toronto09-1176130958.dsl.bell.ca) has joined ##uportal
[11:04:00 CDT(-0500)] <EricDalquist> yay shib delegated auth docs are all done
[11:13:04 CDT(-0500)] <athena> that's great!
[11:13:17 CDT(-0500)] <EricDalquist> it should be really easy to get setup now actually
[11:13:28 CDT(-0500)] <EricDalquist> the web proxy 1.1.1 release includes everything you need
[11:13:38 CDT(-0500)] <EricDalquist> you just have to enable the different http manager
[11:23:52 CDT(-0500)] * colinclark (~colin@142.150.154.130) has joined ##uportal
[11:26:00 CDT(-0500)] * awills (~awills@wsip-72-215-204-133.ph.ph.cox.net) has joined ##uportal
[11:26:01 CDT(-0500)] * bsparks (~bsparks@wsip-72-215-204-133.ph.ph.cox.net) has joined ##uportal
[11:45:55 CDT(-0500)] <awills> if you're asking hibernate for a collection of entities that meet certain criteria because you're planning to remove them, does it make sense to set a querry hint to cache the results? i'm thinking no
[11:46:11 CDT(-0500)] <EricDalquist> correct
[11:46:17 CDT(-0500)] <EricDalquist> and if you're doing clearing in hibernate
[11:46:26 CDT(-0500)] <EricDalquist> it is much faster to use HQL to do a delete directly
[11:46:33 CDT(-0500)] <EricDalquist> instead of using the delete(entity) api
[11:46:53 CDT(-0500)] <EricDalquist> that's really slow unless you happen to already have the entity loaded in the session
[11:48:00 CDT(-0500)] <awills> also... how potentially worrysome is it that each node in a multi-server deployment might be periodically pruning entities, when (clearly) it would only be necessary to have 1 node doing it? would this be a performance or data integrety concern, or is hibernate above that sort of thing?
[11:48:40 CDT(-0500)] <awills> what's the delete hql syntax? was looking at a reference, but didn't immediately see it
[11:48:59 CDT(-0500)] <EricDalquist> http://www.java2s.com/Code/Java/Hibernate/HQLDeleteHQL.htm
[11:49:19 CDT(-0500)] <EricDalquist> it is probably OK ... but some multi-server locking might be a good idea
[11:49:33 CDT(-0500)] <EricDalquist> hibernate can do database level object locking
[11:49:42 CDT(-0500)] <awills> pretty straight forward
[11:49:55 CDT(-0500)] <EricDalquist> but you'd need to create a new hibernate managed object specifically for the lock
[11:50:04 CDT(-0500)] <EricDalquist> thought that might be a decent enhancement
[11:50:18 CDT(-0500)] <EricDalquist> hrm
[11:51:08 CDT(-0500)]

<EricDalquist> create like a: class DistributedLock

Unknown macro: {String lockName; }

[11:51:18 CDT(-0500)] <EricDalquist> define that is a hibernate managed entity
[11:54:52 CDT(-0500)] <EricDalquist> then you can do something like:
[11:54:52 CDT(-0500)] <EricDalquist> DistributedLock purgingLock = session.query("search for lock.lockName=STATS_PURGING_LOCK");
[11:54:52 CDT(-0500)] <EricDalquist> session.lock(purgingLock, LockMode.UPGRADE_NOWAIT);
[11:54:52 CDT(-0500)] <EricDalquist> //Do puring
[11:54:57 CDT(-0500)] <EricDalquist> er purging
[11:55:31 CDT(-0500)] <EricDalquist> if you wrap that in a TX it will release the lock when the TX completes
[11:55:44 CDT(-0500)] <EricDalquist> and you can wrap session.lock with try/catch (LockAcquisitionException lae)
[11:56:00 CDT(-0500)] <EricDalquist> to handle the case where another server already has the lock
[11:57:36 CDT(-0500)] <awills> if your hql is something like this: "DELETE FROM PortalEvent v WHERE v.timeStampAsDate < :timeAsDate"
[11:57:52 CDT(-0500)] <awills> do you think that lock approach would be valuable? or overkill?
[11:58:33 CDT(-0500)] <EricDalquist> I think it could be
[11:58:41 CDT(-0500)] <EricDalquist> I can ask one of our DBAs later today
[11:58:59 CDT(-0500)] <EricDalquist> I'm just not sure if you'll have too much contention
[11:59:13 CDT(-0500)] <EricDalquist> like here if we had 8 servers all running that at the same time
[11:59:23 CDT(-0500)] <EricDalquist> what is the DB going to do
[11:59:48 CDT(-0500)] <EricDalquist> are you going to have a ton of contention for the table?
[11:59:58 CDT(-0500)] <awills> yes, i'm curious too

[12:07:08 CDT(-0500)] <EricDalquist> ok ... back from the dba chat
[12:07:14 CDT(-0500)] <EricDalquist> so we need to do locking
[12:07:17 CDT(-0500)] <EricDalquist> if you have 8 servers
[12:07:25 CDT(-0500)] <EricDalquist> that all run that delete around the same time
[12:07:40 CDT(-0500)] <EricDalquist> the database will do all the delete work for all 8 connections in parallel
[12:07:54 CDT(-0500)] <EricDalquist> the first one that commits will work
[12:08:04 CDT(-0500)] <EricDalquist> the other 7 at that commit point will fail
[12:08:14 CDT(-0500)] <EricDalquist> so functionally it would work
[12:08:25 CDT(-0500)] <EricDalquist> however it would be a huge drain on the database
[12:08:46 CDT(-0500)] <EricDalquist> just to note on the scale of this for large institutions. We generate about 500k events/day
[12:08:46 CDT(-0500)] <awills> got it, will do
[12:08:57 CDT(-0500)] <EricDalquist> we purge anything older than 2 weeks every few minutes
[12:08:59 CDT(-0500)] <awills> roger
[12:09:15 CDT(-0500)] <EricDalquist> a delete of 1 hour of stats can take 30 minutes
[12:09:18 CDT(-0500)] <EricDalquist> :/
[12:09:29 CDT(-0500)] <awills> woah, that's significant
[12:09:33 CDT(-0500)] <EricDalquist> yeah
[12:09:36 CDT(-0500)] <awills> much larger than i imagined
[12:09:45 CDT(-0500)] <EricDalquist> its not a fast operation
[12:09:54 CDT(-0500)] <EricDalquist> the problem is the concurrency
[12:10:07 CDT(-0500)] <EricDalquist> since we're writing stats to those same tables constantly
[12:10:19 CDT(-0500)] <EricDalquist> so the indexes on the tables are in constant flux
[12:10:39 CDT(-0500)] <awills> fwiw i'm adding a 'maxEventAgeMinutes' setting... negative or sero values mean events don't expire, so you could make all servers 0 except 1
[12:10:48 CDT(-0500)] <EricDalquist> yup
[12:10:54 CDT(-0500)] <EricDalquist> and really, a hibernate backed cluster locking API would be good for uPortal
[12:11:08 CDT(-0500)] <EricDalquist> it will be needed whenever we get around to moving stats aggregation into the portal as well
[12:11:21 CDT(-0500)] <EricDalquist> http://docs.jboss.org/hibernate/core/3.3/reference/en/html/transactions.html#transactions-locking
[12:53:09 CDT(-0500)] <awills> EricDalquist – the example for locking you stepped through seems to be in a native Hibernate Java API... can't this task be accomplished in a javax.persistence API? b/c an EntityManager is what we're using so far for this stuff
[12:53:37 CDT(-0500)] <awills> there is EntityManager.lock(Object entity, LockModeType lockMode)
[12:53:42 CDT(-0500)] <awills> seems very similar
[13:29:30 CDT(-0500)] <EricDalquist> right
[13:29:35 CDT(-0500)] <EricDalquist> the problem with that is there is no LOCK_NOWAIT
[13:29:57 CDT(-0500)] <EricDalquist> which you need on the nodes that fail to get the lock so they don't sit there and wait for the purge to complete
[13:30:17 CDT(-0500)] <EricDalquist> was at lunch ... so sorry for the delay
[13:30:47 CDT(-0500)] <awills> no worries
[13:31:08 CDT(-0500)] <EricDalquist> http://uportal.pastebin.com/u4yiDTbU
[13:31:57 CDT(-0500)] <EricDalquist> actually here is the whole class: http://uportal.pastebin.com/ab5yR6fn
[13:32:47 CDT(-0500)] <EricDalquist> and a 'template' style class that makes use of that DAO: http://uportal.pastebin.com/41KXY6Sv
[13:33:01 CDT(-0500)] <awills> k thanks
[13:55:22 CDT(-0500)] <EricDalquist> athena: do you know what the status on https://issues.jasig.org/browse/UP-139 was?
[13:55:32 CDT(-0500)] <EricDalquist> did the customer that wanted it ever get it working?
[13:55:37 CDT(-0500)] <athena> no idea
[14:14:01 CDT(-0500)] <athena> i think it worked, but they were supposed to test import/export
[14:23:30 CDT(-0500)] <EricDalquist> ok
[14:23:40 CDT(-0500)] <EricDalquist> I actually finally have time to work on 3.2.2
[14:23:46 CDT(-0500)] <EricDalquist> so I think I'll apply the patch
[14:28:13 CDT(-0500)] <athena> sounds great
[14:28:38 CDT(-0500)] * lfuller (~sparhk@wsip-72-215-204-133.ph.ph.cox.net) has left ##uportal
[14:41:42 CDT(-0500)] * bsparks (~bsparks@wsip-72-215-204-133.ph.ph.cox.net) has joined ##uportal
[14:58:06 CDT(-0500)] * lfuller (~sparhk@ip68-98-56-21.ph.ph.cox.net) has joined ##uportal
[15:01:56 CDT(-0500)] <EricDalquist> so it makes no sense to minimize a focused portlet right?
[15:02:06 CDT(-0500)] <EricDalquist> you would only minimize something in the '
[15:02:09 CDT(-0500)] <EricDalquist> normal' view
[15:16:37 CDT(-0500)] * lfuller (~sparhk@ip68-98-56-21.ph.ph.cox.net) has joined ##uportal
[15:29:31 CDT(-0500)] <athena> sounds right to me
[15:29:44 CDT(-0500)] <athena> particularly when you're defining minimized as empty
[15:30:10 CDT(-0500)] <athena> were minimized not empty, it might make sense - say if you were using a mobile device or something?
[15:32:26 CDT(-0500)] <EricDalquist> yeah ...
[15:41:46 CDT(-0500)] <awills> EricDalquist could you post the IndexUpdateStatus.java as well?
[15:41:54 CDT(-0500)] <EricDalquist> ah yeah
[15:42:53 CDT(-0500)] <EricDalquist> http://uportal.pastebin.com/ctPyjkdP
[15:43:13 CDT(-0500)] <EricDalquist> the uPortal version of that doesn't need all the fields since it would be just for locking and not for tracking some external process
[15:43:24 CDT(-0500)] <awills> sure thing
[15:43:30 CDT(-0500)] <EricDalquist> it would just need something like a Lock Name so we could use the locking API for different things
[15:43:47 CDT(-0500)] <awills> oh yes, i follow you
[15:44:02 CDT(-0500)] <EricDalquist> sounds like some fun work

[15:53:57 CDT(-0500)] * lfuller (~sparhk@ip68-98-56-21.ph.ph.cox.net) has joined ##uportal
[16:02:35 CDT(-0500)] <awills> for a hibernate-managed object, fo all fields require setters? (i cen't remember)
[16:02:57 CDT(-0500)] <awills> can i just provide a constructor DistributedLock(String name)
[16:03:04 CDT(-0500)] <EricDalquist> no
[16:03:11 CDT(-0500)] <EricDalquist> hibernate does method level variable access
[16:03:32 CDT(-0500)] <EricDalquist> what we've done for other DAOs is define the data object with an interface
[16:03:40 CDT(-0500)] <EricDalquist> and make it so only the DAO can create the managed impl
[16:03:47 CDT(-0500)] <awills> gotcha
[16:03:53 CDT(-0500)] <EricDalquist> that protects the object from being created but not actually persisted
[16:04:06 CDT(-0500)] <EricDalquist> you could even do on-demand with these
[16:04:12 CDT(-0500)] <EricDalquist> getDistributedLock(String name)
[16:04:16 CDT(-0500)] <EricDalquist> which will get or create
[16:04:23 CDT(-0500)] <awills> yeah i have that
[16:04:55 CDT(-0500)] <awills> http://uportal.pastebin.com/psTnsEQ4
[16:05:05 CDT(-0500)] <awills> based on your example
[16:05:19 CDT(-0500)] <EricDalquist> very nice

[16:05:50 CDT(-0500)] <EricDalquist> though I still find it annoying that doing that logic flow is so ugly
[16:09:19 CDT(-0500)] <awills> heh... it's a perfect parrot of the "double-check synchronization" idiom also
[16:09:31 CDT(-0500)] <EricDalquist> yup
[16:10:00 CDT(-0500)] <EricDalquist> though that does work in JDK5+ if your double-checked field is volatile
[16:10:02 CDT(-0500)] <awills> so let me guess: i need a property marked @Version to use an instance as a lock?
[16:10:11 CDT(-0500)] <EricDalquist> I don't think so
[16:10:23 CDT(-0500)] <EricDalquist> we just had that there to track the number of index updates that had happened
[16:11:50 CDT(-0500)] <awills> javadoc for EntityManager: "The persistence implementation is not required to support calling EntityManager.lock() on a non-versioned object"
[16:12:05 CDT(-0500)] <EricDalquist> ah
[16:12:14 CDT(-0500)] <awills> i know we're not using that API, but i was suspecting there was a corresponding hbm restriction
[16:13:13 CDT(-0500)] <EricDalquist> looking at: http://docs.jboss.org/hibernate/core/3.3/reference/en/html/transactions.html#transactions-locking
[16:13:18 CDT(-0500)] <EricDalquist> Session.lock() performs a version number check if the specified lock mode is READ, UPGRADE or UPGRADE_NOWAIT. In the case of UPGRADE or UPGRADE_NOWAIT, SELECT ... FOR UPDATE is used.
[16:48:42 CDT(-0500)] <awills> EricDalquist – if you have a sec to make any early comments: http://uportal.pastebin.com/4cgFLkM4
[16:49:03 CDT(-0500)] <awills> i haven't wired it in the context or attempted to make it work yet
[16:50:42 CDT(-0500)] <EricDalquist> looks good, the only changes I would make would be to make the constructors on DistributedLock default scope and add equals/hashcode/tostring methods (eclipse can generate them) so it's a fully realized bean.
[16:50:48 CDT(-0500)] <EricDalquist> got to run ... see everyone tomorrow
[16:51:41 CDT(-0500)] <awills> cya
[17:00:19 CDT(-0500)] * JoeMoore (~irchon@24.220.31.20) has joined ##uportal

Browser not supported