Versions Compared

Old Version 51

changes.mady.by.user Apereo Infrastructure

Saved on

compared with

New Version 52

changes.mady.by.user Apereo Infrastructure

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

[09:07:36 CDT(-0500)] <EricDalquist> athena: do you foresee any problems with me batch converting all of the uPortal source code to UTF-8?
[09:34:02 CDT(-0500)] <athena> other than tons of conflicts in svn stuff i've created? no. (smile)
[09:34:38 CDT(-0500)] <EricDalquist> (smile)
[09:34:45 CDT(-0500)] <EricDalquist> well of course I'd ask for people to sync up first
[09:34:59 CDT(-0500)] <athena> well, that only helps for direct checkouts
[09:35:02 CDT(-0500)] <athena> but sounds like a good idea
[09:35:07 CDT(-0500)] <EricDalquist> hrm
[09:35:08 CDT(-0500)] <athena> and if we're going to do it, now sounds like a great time
[09:35:08 CDT(-0500)] <EricDalquist> true
[09:35:23 CDT(-0500)] <EricDalquist> just trying to figure out how to fix the encoding issues during compiling people are hitting
[09:35:27 CDT(-0500)] <athena> likely few people other than me have imported 4.0 as a vendor drop
[09:35:30 CDT(-0500)] <athena> so i think we should do it
[09:35:43 CDT(-0500)] <athena> is that something we can enforce in git?
[09:36:05 CDT(-0500)] <EricDalquist> not sure
[09:36:07 CDT(-0500)] <EricDalquist> I'll look into it
[09:56:48 CDT(-0500)] <athena> so what's going on w/ the data import now?
[09:57:02 CDT(-0500)] <athena> looks like just some of the groups are in default and the others are in quickstart?
[09:57:17 CDT(-0500)] <EricDalquist> yeah
[09:57:28 CDT(-0500)] <EricDalquist> I tried to move the minimal set of data into default
[09:57:35 CDT(-0500)] <EricDalquist> so that the import works at each stage
[09:57:38 CDT(-0500)] <athena> what's the order of import? is it all of default, then all of quickstart?
[09:58:50 CDT(-0500)] <EricDalquist> required, default, quickstart
[09:59:08 CDT(-0500)] <athena> is that what we want, or do we actually want to do each of those in turn by type?
[09:59:32 CDT(-0500)] <EricDalquist> I'm not sure I follow
[10:00:21 CDT(-0500)] <athena> so instead of importing all default entitities, then all quickstart entities
[10:00:41 CDT(-0500)] <athena> just wondering if we should import all default groups, then all quickstart groups, then all default portlets, etc.
[10:00:54 CDT(-0500)] <EricDalquist> ah
[10:01:22 CDT(-0500)] <EricDalquist> that could be doable, just a little work to allow batch import from multiple directories
[10:01:34 CDT(-0500)] <EricDalquist> are you worried about the couple of duplicate groups?
[10:01:59 CDT(-0500)] <EricDalquist> the problem is I'd like the data to be valid out of the box for someone that just wants to import required or just required & default
[10:02:05 CDT(-0500)] <athena> yeah that makes sense (smile)
[10:02:08 CDT(-0500)] <athena> probalby best as is
[10:04:15 CDT(-0500)] <EricDalquist> hrm
[10:04:24 CDT(-0500)] <EricDalquist> I'm not finding any files with mixed encodings
[10:04:43 CDT(-0500)] <athena> that's good
[10:04:47 CDT(-0500)] <EricDalquist> either that or my command line fu isn't working right
[13:31:43 CDT(-0500)] <athena> hey EricDalquist
[13:31:51 CDT(-0500)] <athena> you said something about nick finishing support for resource urls?
[13:31:52 CDT(-0500)] <EricDalquist> hi athena
[13:31:56 CDT(-0500)] <EricDalquist> yes
[13:31:58 CDT(-0500)] <EricDalquist> that should be there
[13:32:06 CDT(-0500)] <athena> any examples of it?
[13:32:12 CDT(-0500)] <athena> not having much initial success
[13:32:18 CDT(-0500)] <EricDalquist> ah I think the functional test portlet in svn does it
[13:32:23 CDT(-0500)] <EricDalquist> just asked nick to join
[13:32:27 CDT(-0500)] <athena> awesome (smile)
[13:32:36 CDT(-0500)] <EricDalquist> I've spent most of my day trying not to smash my computer
[13:32:42 CDT(-0500)] <EricDalquist> at home now so we'll see if my laptop behaves better
[13:32:58 CDT(-0500)] <athena> eep (smile)
[13:34:31 CDT(-0500)] <athena> hmm, might just be that i haven't set an expiration timeout
[13:34:34 CDT(-0500)] <nickblair> hello - what's up?
[13:34:36 CDT(-0500)] <athena> does one have to be set?
[13:34:48 CDT(-0500)] <nickblair> for cachecontrol - absolutely
[13:34:51 CDT(-0500)] <athena> gotcha
[13:34:56 CDT(-0500)] <athena> i'm just not sure what to set it to
[13:34:57 CDT(-0500)] <nickblair> even if one is using validation (etag) method
[13:35:00 CDT(-0500)] <athena> what does that mean practically?
[13:35:07 CDT(-0500)] <nickblair> something greater than 0
[13:35:09 CDT(-0500)] <athena> the next time it checks the etag or ?
[13:35:14 CDT(-0500)] <nickblair> it means the number of seconds
[13:35:20 CDT(-0500)] <nickblair> that the element is valid for
[13:35:22 CDT(-0500)] <EricDalquist> that is how long the portal will cache the content for in memory
[13:35:49 CDT(-0500)] <athena> ok
[13:36:02 CDT(-0500)] <nickblair> playing with new cachecontrol support for resource urls?
[13:36:10 CDT(-0500)] <athena> and then when it expires, the portlet has another chance to just say "eh, whatever, the etag still matches, lets set it again"?
[13:36:12 CDT(-0500)] <athena> yeah
[13:36:15 CDT(-0500)] <nickblair> yes
[13:36:22 CDT(-0500)] <athena> cool, thanks (smile)
[13:36:26 CDT(-0500)] <EricDalquist> wait
[13:36:48 CDT(-0500)] <EricDalquist> I think when the cached content expires from the portal's the portlet won't get a chance to validate the etag
[13:36:50 CDT(-0500)] <EricDalquist> with an exception
[13:37:03 CDT(-0500)] <EricDalquist> since we're passing the etag to the browser for resources
[13:37:04 CDT(-0500)] <nickblair> check the tests in the FunctionalTestsPortlet - there are 2 examples
[13:37:15 CDT(-0500)] <EricDalquist> it doesn't have to be in the portal cache for etag validation to actually work
[13:37:27 CDT(-0500)] <nickblair> right
[13:37:36 CDT(-0500)] <athena> oh ok
[13:37:42 CDT(-0500)] <EricDalquist> for rendered content once it expires from the portal's cache the portlet can't revalidate the etag
[13:37:46 CDT(-0500)] <EricDalquist> because there is nothing to reply
[13:37:47 CDT(-0500)] <EricDalquist> replay
[13:38:32 CDT(-0500)] <athena> so where's an example of just using Etags?
[13:38:34 CDT(-0500)] <EricDalquist> also ... I didn't realize the expiration time was required with etags
[13:38:43 CDT(-0500)] <EricDalquist> is that from the spec doc nickblair?
[13:38:48 CDT(-0500)] <nickblair> that is from the spec doc
[13:38:49 CDT(-0500)] <athena> what we're really looking for in these portlets is browser-side caching
[13:38:57 CDT(-0500)] <nickblair> use validation
[13:39:10 CDT(-0500)] <nickblair> one sec, i'll get a link to point to examples
[13:39:15 CDT(-0500)] <athena> thanks
[13:39:33 CDT(-0500)] <EricDalquist> right and I think that should be doable now by setting an etag with an expiration time
[13:39:35 CDT(-0500)] <nickblair> first is the example direct from the spec:
[13:39:36 CDT(-0500)] <nickblair> https://source.jasig.org/portlets/FunctionalTestsPortlet/trunk/src/main/java/org/jasig/portlet/test/cachecontrol/publicscope/CacheControlTestPortlet.java
[13:39:41 CDT(-0500)] <nickblair> extends GenericPortlet
[13:39:52 CDT(-0500)] <nickblair> that's for render though
[13:39:55 CDT(-0500)] <EricDalquist> and the portlet will be given a chance to validate the etag even if the portal doesn't have the content cached in memory
[13:40:08 CDT(-0500)] <nickblair> 2nd example: https://source.jasig.org/portlets/FunctionalTestsPortlet/trunk/src/main/java/org/jasig/portlet/test/mvc/tests/CacheControlTestController.java
[13:40:13 CDT(-0500)] <nickblair> that uses resource urls
[13:40:22 CDT(-0500)] <athena> hm, ok
[13:40:27 CDT(-0500)] <nickblair> note the comment:
[13:40:28 CDT(-0500)] <nickblair> // have to set an expiration time when using etag per portlet spec (empty expiration time is treated as "expired")
[13:40:56 CDT(-0500)] <nickblair> in writeJsonContentWithValidationCaching
[13:41:13 CDT(-0500)] <athena> not sure i'm still quite following how this works
[13:41:21 CDT(-0500)] <athena> so when the content expires in the portal cache, what happens?
[13:41:30 CDT(-0500)] <athena> does it not send the ETag through w/t he request anymore?
[13:41:32 CDT(-0500)] <nickblair> depends on if it's a render or a resource
[13:41:41 CDT(-0500)] <athena> for a resource, specifically
[13:42:54 CDT(-0500)] <EricDalquist> for a resource request, if the browser sends an etag the portal will pass it along to the portlet even if there is nothing cached in memory for that request
[13:43:03 CDT(-0500)] <athena> ok, perfect
[13:43:07 CDT(-0500)] <athena> that's what i was hoping was true (smile)
[13:43:11 CDT(-0500)] <nickblair> i'm not sure that's the case, could be a bug
[13:43:19 CDT(-0500)] <nickblair> do you have an example you are working with now?
[13:43:21 CDT(-0500)] <athena> that's definitely the behavior we want, right?
[13:43:24 CDT(-0500)] <EricDalquist> that should be how it works at least (wink)
[13:43:27 CDT(-0500)] <athena> not yet - trying to get something working
[13:43:30 CDT(-0500)] <nickblair> ok
[13:43:43 CDT(-0500)] <athena> for a resource url, what exactly does the expiration time mean?
[13:43:58 CDT(-0500)] <nickblair> i think if there is cached content but it's expired, it'll trigger portletContainer#doServeResource again
[13:44:10 CDT(-0500)] <nickblair> if your portlet sees the etag and determines its still valid
[13:44:12 CDT(-0500)] <EricDalquist> I'll go re-read the spec
[13:44:14 CDT(-0500)] <EricDalquist> just so we're sure
[13:44:14 CDT(-0500)] <athena> gotcha
[13:44:18 CDT(-0500)] <nickblair> and your portlet says "useCachedContent(true"
[13:44:19 CDT(-0500)] <EricDalquist> ah
[13:44:22 CDT(-0500)] <nickblair> then we return 304
[13:44:35 CDT(-0500)] <EricDalquist> so if you set an etag and an expiration time of 60
[13:44:38 CDT(-0500)] <nickblair> the confusing part here is that if you are using validation caching
[13:44:44 CDT(-0500)] <nickblair> your portlet gets a chance to run
[13:44:47 CDT(-0500)] <EricDalquist> the portal won't even let you do etag validation for 60 seconds?
[13:45:05 CDT(-0500)] <nickblair> if you want to return whats cached, you say useCachedContent(true)
[13:45:23 CDT(-0500)] <nickblair> yes eric, portlet will let you do etag validation for 60
[13:45:34 CDT(-0500)] <nickblair> but since your doing etag
[13:45:44 CDT(-0500)] <nickblair> you get the opportunity to keep saying useCachedContent(true)
[13:46:00 CDT(-0500)] <nickblair> and with resources, if you do that, you'll keep returning 304s
[13:46:09 CDT(-0500)] <EricDalquist> so if you set an etag, the number you set for the expiration has no effect
[13:46:12 CDT(-0500)] <EricDalquist> other than it has to be set
[13:46:13 CDT(-0500)] <EricDalquist> ?
[13:46:21 CDT(-0500)] <nickblair> as long as the client sends the same Etag back
[13:46:45 CDT(-0500)] <nickblair> if the client for whatever doesnt send the tag, and the data is expired - we've got a potential bug
[13:47:11 CDT(-0500)] <nickblair> take a look at PortletRendererImpl doServeResource and doServeResourceCachedOutput
[13:47:13 CDT(-0500)] <EricDalquist> wait ... here is what the specification says
[13:48:01 CDT(-0500)] <EricDalquist> After the content is expired the portlet container should send a render or serveResource request to the portlet with the validation token (called ETag in HTTP) of the expired content. The portlet can access the validation token provided by the portlet container either via the property ETAG of the RenderRequest or ResourceRequest, or the getETag method of the RenderRequest or ResourceRequest. The portlet can validate i
[13:48:05 CDT(-0500)] <EricDalquist> ----------
[13:48:17 CDT(-0500)] <EricDalquist> So, when you set an etag, you have to set an expiration time
[13:48:21 CDT(-0500)] <nickblair> -1 is forever, 0 is expired, >0 is number of seconds
[13:48:33 CDT(-0500)] <EricDalquist> the expiration time you set is the duration for which the portal will return that cached content and NOT consult the portlet
[13:48:49 CDT(-0500)] <EricDalquist> once the expiration time elapses the portal should consult the porltet with the etag
[13:49:01 CDT(-0500)] <EricDalquist> and determine if the expirted content should be returned or not
[13:49:11 CDT(-0500)] <EricDalquist> that is what the spec requires
[13:49:40 CDT(-0500)] <nickblair> the only thing that may be wrong with doServeResource then is the first test in doServeResource where it checks for a null CacheControl#etag
[13:50:57 CDT(-0500)] <nickblair> when I interpreted the spec I thought it was 2 passes - you only looked at expiration time if etag wasn't set, but expiration time was required regardless
[13:51:19 CDT(-0500)] <nickblair> the expiration time is passed through into CacheControlsServiceImpl as one of the arguments to the cache Element
[13:51:23 CDT(-0500)] <EricDalquist> can you be more specific with that "you"
[13:51:47 CDT(-0500)] <nickblair> "you" should mean the PortletRenderer render and resource methods
[13:52:14 CDT(-0500)] <EricDalquist> so I feel like the spec is pretty clear on the steps:
[13:52:29 CDT(-0500)] <EricDalquist> 1. when a portlet renders it sets an etag AND an expiration time
[13:53:11 CDT(-0500)] <EricDalquist> 2. when a request comes in and there is a portal side cache entry that matches the first check is to see if the expiration time has passed
[13:53:24 CDT(-0500)] <EricDalquist> 3a. if the expiration time has not passed the content is replaye
[13:53:46 CDT(-0500)] <EricDalquist> 3b. if the expiration time has passed the portlet is executed and provided the etag of the cached content
[13:54:22 CDT(-0500)] <EricDalquist> 4. the portlet can look at the etag and if it is still a valid etag it tells the portal to used the cached content and sets a new expiration time
[13:54:29 CDT(-0500)] <EricDalquist> or it renders new content
[13:54:35 CDT(-0500)] <EricDalquist> ----------
[13:54:37 CDT(-0500)] <nickblair> reading 3b, it seems like we need to keep entries in the cache for longer than expirationTime.
[13:54:40 CDT(-0500)] <EricDalquist> yes
[13:54:42 CDT(-0500)] <EricDalquist> we do
[13:54:49 CDT(-0500)] <EricDalquist> expiration time is just an attribute of cached content
[13:54:54 CDT(-0500)] <EricDalquist> it isn't when we actually expire them
[13:55:27 CDT(-0500)] <EricDalquist> really etag'd entries should only expire when the cache is full
[13:55:32 CDT(-0500)] <nickblair> PortletCacheControlServiceImpl needs some changes then - simple enough, we can just default to whatever the ehcache.xml config is (use the constructor Element(key, value)
[13:55:59 CDT(-0500)] <EricDalquist> for portlets that cache using ONLY expiration time the cache element should use the expirationTime as the cache expiration timeout
[13:56:09 CDT(-0500)] <EricDalquist> since those we just want to go away
[13:56:12 CDT(-0500)] <nickblair> and PortletRendererImpl should skip the check for null etag in the first cached content test
[13:56:16 CDT(-0500)] <EricDalquist> but even then we should probably be smart about it
[13:56:31 CDT(-0500)] <EricDalquist> and only use the portlet provided expiration time if it is less than the cache's configured TTL
[13:56:41 CDT(-0500)] <nickblair> i've got to run
[13:56:46 CDT(-0500)] <EricDalquist> ok
[13:56:49 CDT(-0500)] <athena> thanks (smile)
[13:56:54 CDT(-0500)] <athena> going to keep poking at this
[13:56:59 CDT(-0500)] <nickblair> we can chat tomorrow more
[13:57:04 CDT(-0500)] <athena> sounds good!
[13:57:04 CDT(-0500)] <nickblair> i'd love to see more use cases
[13:57:07 CDT(-0500)] <EricDalquist> yup
[13:57:18 CDT(-0500)] <nickblair> later!
[13:59:23 CDT(-0500)] <EricDalquist> only 106 files with bad encoding or line endings athena
[13:59:53 CDT(-0500)] <athena> only!
[13:59:54 CDT(-0500)] <athena> (tongue)
[14:00:05 CDT(-0500)] <athena> we need to start enforcing this stuff
[14:00:13 CDT(-0500)] <EricDalquist> yes
[14:02:42 CDT(-0500)] <athena> ok, i'll be back in a few - lunchtime
[14:20:36 CDT(-0500)] <athena> back
[15:22:58 CDT(-0500)] <EricDalquist> https://wiki.jasig.org/display/UPC/JSR-286%20Caching%20Use%20Cases
[15:23:01 CDT(-0500)] <EricDalquist> how does that look athena
[15:23:38 CDT(-0500)] <athena> will read through
[15:23:56 CDT(-0500)] <EricDalquist> bah
[15:24:03 CDT(-0500)] <EricDalquist> just realized I skipped some stuff
[15:24:07 CDT(-0500)] <EricDalquist> so don't read yet
[15:24:39 CDT(-0500)] <athena> lol ok
[15:26:04 CDT(-0500)] <EricDalquist> never mind
[15:26:08 CDT(-0500)] <EricDalquist> I just can't read (tongue)
[15:26:25 CDT(-0500)] <athena> lol
[15:27:19 CDT(-0500)] <EricDalquist> though I am going to make a few more changes quickly here
[15:27:20 CDT(-0500)] <athena> this looks good to me
[15:38:27 CDT(-0500)] <athena> ahhhhhh this hasn't been working because i was deploying the wrong portlet
[15:38:30 CDT(-0500)] <athena> smacks head
[15:38:37 CDT(-0500)] <EricDalquist> lol
[15:38:41 CDT(-0500)] <EricDalquist> I hate that
[15:38:45 CDT(-0500)] <athena> yes!
[15:38:46 CDT(-0500)] <EricDalquist> too many terminal tab (wink)
[15:38:47 CDT(-0500)] <athena> so frustrating
[15:38:57 CDT(-0500)] <athena> any bug that takes a while to figure out is always something stupid
[15:39:04 CDT(-0500)] <EricDalquist> need like a mvn script that sets the tab title to the project name
[15:39:17 CDT(-0500)] <athena> hehe
[15:47:23 CDT(-0500)] <EricDalquist> I think the encoding issues those people reported were from xjc
[15:49:09 CDT(-0500)] <athena> ah
[15:49:22 CDT(-0500)] <EricDalquist> it was adding a class level commet
[15:49:30 CDT(-0500)] <EricDalquist> that contained localized characters
[15:49:40 CDT(-0500)] <EricDalquist> but the files are generated in ascii
[15:49:48 CDT(-0500)] <EricDalquist> and xjc apparently has no encoding option
[15:50:14 CDT(-0500)] <athena> ahh
[15:50:16 CDT(-0500)] <athena> gotcha
[15:50:25 CDT(-0500)] <EricDalquist> http://java.net/jira/browse/JAXB-499
[16:31:11 CDT(-0500)] <athena> i'm really not having much luck w/ the caching stuff
[16:31:19 CDT(-0500)] <athena> though it seems to work fine for the func test portlet
[16:31:23 CDT(-0500)] <EricDalquist> hrm
[16:31:26 CDT(-0500)] <athena> wondering if there's something different about spring portlet mvc?
[16:31:49 CDT(-0500)] <EricDalquist> the spring MVC may not actually support invalidation caching correctly
[16:31:58 CDT(-0500)] <EricDalquist> there are lots of hole in the portlet 2 support :/
[16:32:06 CDT(-0500)] <EricDalquist> hopefully nick can provide some input tomorrow morning
[16:32:11 CDT(-0500)] <EricDalquist> he's golfing all day today (tongue)
[16:34:33 CDT(-0500)] <athena> lol awesome
[16:34:39 CDT(-0500)] <athena> sounds like a fun day (tongue)
[16:45:02 CDT(-0500)] <dustin_> Hey EricDalquist is there somewhere on the uportal wiki that documents the process of using the maven release plugin to do uportal releases?
[16:45:18 CDT(-0500)] <EricDalquist> https://wiki.jasig.org/display/UPC/Cutting+a+uPortal+Release
[16:45:22 CDT(-0500)] <EricDalquist> (smile)
[16:45:35 CDT(-0500)] <EricDalquist> though be a little careful since that assumes a uPortal release
[16:45:39 CDT(-0500)] <EricDalquist> and I'm assuming you're doing umobile?
[16:46:31 CDT(-0500)] <athena> hey dustin_
[16:46:47 CDT(-0500)] <dustin_> hey hey
[16:46:57 CDT(-0500)] <EricDalquist> back in ~ 10 min
[16:47:12 CDT(-0500)] <dustin_> do you use the release plugin in release branches?
[16:48:04 CDT(-0500)] <athena> are you talking about cutting a release of like an institutional version of uportal?
[16:48:54 CDT(-0500)] <dustin_> more about cutting a bug fix release from a patches branch
[16:49:21 CDT(-0500)] <athena> i think eric does use the release plugin for those
[16:49:39 CDT(-0500)] <athena> and i was able to successfully cut a umobile release, where the current umobile trunk is a branch off the uportal project
[16:50:00 CDT(-0500)] <dustin_> ok
[16:51:12 CDT(-0500)] <EricDalquist> back
[16:51:19 CDT(-0500)] <EricDalquist> yes
[16:51:22 CDT(-0500)] <EricDalquist> I do use the release plugin
[16:51:24 CDT(-0500)] <dustin_> so what about creating a new minor release in the trunk? do you use the release plugin there?
[16:51:34 CDT(-0500)] <EricDalquist> yes
[16:51:55 CDT(-0500)] <EricDalquist> are you volunteering to cut a patch release of uPortal? (wink)
[16:52:02 CDT(-0500)] <dustin_> god no
[16:52:07 CDT(-0500)] <EricDalquist> awe
[16:52:11 CDT(-0500)] <EricDalquist> its not hard (sad)
[16:52:19 CDT(-0500)] <athena> lol
[16:52:28 CDT(-0500)] <athena> we coudl really use a 3.2.x release . . . (smile)
[16:52:28 CDT(-0500)] <dustin_> just trying to figure out a similar releasing strategy
[16:54:27 CDT(-0500)] <athena> hmm does -DautoVersionSubmodules save you from having to hit enter 10 million times? (smile)
[16:54:34 CDT(-0500)] <EricDalquist> yes
[16:54:35 CDT(-0500)] <EricDalquist> it does
[16:54:40 CDT(-0500)] <athena> good to know
[16:54:43 CDT(-0500)] <EricDalquist> (smile)
[16:54:43 CDT(-0500)] <athena> that was a pain (smile)
[16:54:47 CDT(-0500)] <EricDalquist> yes it is