uPortal IRC Logs-2011-02-03

[09:30:20 CST(-0600)] <holdorph> with great power comes great responsibility....
[10:05:33 CST(-0600)] <rickard> athena: You awake?
[10:05:40 CST(-0600)] <athena> yes
[10:06:12 CST(-0600)] <rickard> I've looked some more at the utf8 issues today and I'm pretty convinced it's not a CKEditor issue.
[10:06:36 CST(-0600)] <rickard> I still think you should set:
[10:06:36 CST(-0600)] <rickard> config.entities = false;
[10:06:37 CST(-0600)] <rickard> config.entities_greek = false;
[10:06:37 CST(-0600)] <rickard> config.entities_latin = false;
[10:06:44 CST(-0600)] <rickard> in config.js though.
[10:08:02 CST(-0600)] <rickard> My reasoning is this. The data is garbled when hitting save in the portlet manager. Not when hitting save when editing the rich configiration. When you hit that save button, you're redirected back to the main portlet manager page and there, in the preview, UTF8 characters are fine.
[10:09:07 CST(-0600)] <rickard> I also doubt it's a tomcat issue because then we would have problems with other portlets or with the uportal admin interface in general. We don't.
[10:09:36 CST(-0600)] <athena> hm.
[10:10:00 CST(-0600)] <athena> so actually when you hit "save" in the portlet config mode it does the save right then
[10:10:06 CST(-0600)] <athena> the main portlet manager doesn't do anything
[10:10:16 CST(-0600)] <rickard> Hmm, that's odd.
[10:10:23 CST(-0600)] <athena> so it's showing up in the review page ok?
[10:10:27 CST(-0600)] <rickard> Yup
[10:10:34 CST(-0600)] <rickard> Until you exit that screen and go back.
[10:11:06 CST(-0600)] <athena> wow
[10:11:12 CST(-0600)] <athena> ok, we'll have to think about that one
[10:11:22 CST(-0600)] <athena> i'm really not sure what would cause that
[10:11:32 CST(-0600)] <rickard> Give me a second to test something.
[10:16:29 CST(-0600)] <rickard> Yes, that's what happens. Just wanted to confirm.
[10:20:57 CST(-0600)] <rickard> athena: I haven't checked with any other portlets that offer rich configuration, but could it perhaps be a problem with uPortal and not the portlet? Perhaps UTF8 is not supported for portlet preferences.
[10:21:33 CST(-0600)] <athena> i suppose it's possible that there's some issue w/ portlet prefs, but if there were i'd expect it to show up on the review screen
[10:21:43 CST(-0600)] <rickard> True
[10:21:56 CST(-0600)] <rickard> Can you confirm the behaviour on your end?
[10:22:31 CST(-0600)] <rickard> http://www.utf8-chartable.de/unicode-utf8-table.pl
[10:22:40 CST(-0600)] <rickard> If you need some exotic characters to test with
[10:23:27 CST(-0600)] <athena> thanks!
[10:23:33 CST(-0600)] <athena> i can't do it this morning, but i'll try and take a look later
[10:23:52 CST(-0600)] <athena> i'll have to do it in 3.2.x - no config mode in trunk
[10:24:02 CST(-0600)] <rickard>
[10:24:02 CST(-0600)] * athena is sadly bogged down in work tasks these days
[10:24:11 CST(-0600)] * holdorph gasps
[10:24:18 CST(-0600)] <rickard> Must be a pain to have to deal with the "old stuff", right
[10:24:18 CST(-0600)] <athena> haha
[10:24:25 CST(-0600)] <athena> no it's not actually old
[10:24:39 CST(-0600)] <athena> it's just that config mode in trunk isn't working right now, so it's not a good place to test this
[10:25:03 CST(-0600)] <athena> planning to ship this portlet in the next major rev of uportal, so it would be nice to get these details fixed
[10:25:12 CST(-0600)] <rickard> Yeah
[10:25:45 CST(-0600)] <athena> trying to improve i18n in the next version of uportal
[10:25:52 CST(-0600)] <athena> so this would fit in well with that too
[10:25:53 CST(-0600)] <rickard> You wouldn't happen to know where in the uportal source the portlet preferences are persisted, would you?
[10:25:58 CST(-0600)] <rickard> I could have a look myself.
[10:26:20 CST(-0600)] <athena> there are a bunch of JPA classes - shouldn't be too hard to find
[10:26:42 CST(-0600)] <athena> in 3.2 there's an IPortletDefinition/PortletDefinitionImpl object that has PortletPreference objects (or something like that)
[10:26:54 CST(-0600)] <athena> and then there are obviously-named classes like JpaPortletDefinitionDao
[10:27:12 CST(-0600)] <rickard> Ok, I'll have a look around for anything suspicious.
[10:28:03 CST(-0600)] <athena> maybe in theory it's possible a final save to the channel object cascades down and affects the portlet prefs?
[10:28:07 CST(-0600)] <athena> could check on that anyway
[10:29:03 CST(-0600)] <rickard> Yup. Sounds a bit over my head though. I'll leave that in your capable hands
[12:14:05 CST(-0600)] <EricDalquist> thinking like a portlet developer ... is it your responsibility or the portal's to deal with concurrent modification of the PortletPreferences object
[12:15:22 CST(-0600)] <EricDalquist> guess the 2.0 spec clarifies that actually "The portlet container implementation is responsible for handling concurrent writes to avoid inconsistency in portlet preference attributes"
[12:48:22 CST(-0600)] <EricDalquist> lfuller: have you ever dealt with concurrent modification of hibernate managed objects?
[12:50:03 CST(-0600)] <rickard> athena: I just tried naming a tab using a character from the Cyrillic alphabet and it works as long as I stay logged in. If I log out and in again, the character is replaced with a question mark. In other words, this is most likely an issue with uPortal and not the portlet.
[12:50:55 CST(-0600)] <athena> were you able to switch the db over to utf8?
[12:51:06 CST(-0600)] <rickard> Yes, it's done.
[12:51:43 CST(-0600)] <rickard> That is unless uPortal has stored the old collation somewhere and it needs an initportal to pick up the change.
[12:54:07 CST(-0600)] <EricDalquist> I'd assume you have to recreate the tables to pick up the new encoding
[12:54:22 CST(-0600)] <EricDalquist> but I know very little about mysql
[12:54:42 CST(-0600)] <rickard> As far as MySQL goes, you can switch the encoding.
[12:55:03 CST(-0600)] <EricDalquist> and that affects the content of already existing tables and data?
[12:55:22 CST(-0600)] <rickard> Yes, if I'm reading the manual correctly.
[12:55:25 CST(-0600)] <rickard> ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name
[12:56:33 CST(-0600)] <rickard> Regardless, the encoding is changed so any future data should use the correct encoding.
[12:58:25 CST(-0600)] <lfuller> eric, you running into a situation where you have some code that has grabbed a hibernate entity, and is not getting updates to the same entity in a different context?
[12:58:44 CST(-0600)] <EricDalquist> well I think I may have found the issue
[12:58:50 CST(-0600)] <EricDalquist> it is dealing with portlet preferences
[12:58:58 CST(-0600)] <EricDalquist> and it may actually be related to the iterim entity stuff
[12:59:08 CST(-0600)] <lfuller> k
[12:59:12 CST(-0600)] <EricDalquist> if 2 threads both go to store preferences on an intrim entityt
[12:59:19 CST(-0600)] <EricDalquist> I get a contstraint failure
[12:59:56 CST(-0600)] <EricDalquist> I think I need to add some locking around the preferences store method and then the part I just realized is that I should refresh my entity object from the DAO inside the lock
[13:00:09 CST(-0600)] <EricDalquist> to make sure I'm working with whatever the most recently stored version is
[13:04:30 CST(-0600)] <EricDalquist> yay that was it!
[13:15:45 CST(-0600)] <EricDalquist> so lfuller any ideas how to deal with locking for portlet entities
[13:16:15 CST(-0600)] <EricDalquist> I'm realizing that there really needs to be some basic areas around portlet preferences and the conversion to/from interim entities that need locking
[13:16:43 CST(-0600)] <EricDalquist> I can't really do JPA backed DB locks since part of the time there is no DB backed entity object
[13:17:33 CST(-0600)] <EricDalquist> my only thought right now is to track one Lock per entity object in the user's session
[13:17:45 CST(-0600)] <EricDalquist> which should work in most cases
[13:17:50 CST(-0600)] <EricDalquist> but could fail if a user is logged in twice
[13:21:29 CST(-0600)] <rickard> uPortal uses hibernate, right?
[13:21:47 CST(-0600)] <rickard> Just found this: http://www.isocra.com/2007/01/utf-8-with-hibernate-30-and-mysql/
[13:21:47 CST(-0600)] <EricDalquist> yes
[13:23:55 CST(-0600)] <rickard> And this: if you are using EJB3/JPA, add to persistence.xml the following properties (in the appropriate <persistence-unit>):
[13:23:55 CST(-0600)] <rickard> <property name="hibernate.connection.useUnicode" value="true" />
[13:23:56 CST(-0600)] <rickard> <property name="hibernate.connection.characterEncoding" value="UTF-8" />
[13:24:12 CST(-0600)] <rickard> Any idea if those are set?
[13:24:27 CST(-0600)] <EricDalquist> not sure
[13:24:39 CST(-0600)] <EricDalquist> uportal uses JPA with Hibernate as the underlying impl
[13:25:06 CST(-0600)] <EricDalquist> looks like no:
[13:25:06 CST(-0600)] <EricDalquist> <property name="hibernate.ejb.cfgfile" value="/properties/db/hibernate.cfg.xml" />
[13:25:06 CST(-0600)] <EricDalquist> <property name="hibernate.archive.autodetection" value="" />
[13:25:34 CST(-0600)] <rickard> I'll give it a try.
[14:15:50 CST(-0600)] <rickard> athena, EricDalquist: I found the problem. Adding useUnicode=true&characterEncoding=UTF-8 to the end of hibernate.connection.url in rdbm.properties. This fixes both UTF8 in SimpleContentPortlet as well as e.g. naming a tab using UTF8 characters.
[14:16:45 CST(-0600)] <EricDalquist> that would be good to add to the mysql bit of the manual
[14:17:04 CST(-0600)] <rickard> Indeed. Or have it by default in the stock rdbp.properties
[14:17:20 CST(-0600)] <rickard> There's no need to not use UTF-8, even for you guys.
[14:17:22 CST(-0600)] <EricDalquist> in the mysql example?
[14:17:25 CST(-0600)] <rickard> Yeah
[14:17:37 CST(-0600)] <EricDalquist> yeah that would be good
[14:17:47 CST(-0600)] <EricDalquist> could you create a jira issue for the rdbm.properties task?
[14:17:51 CST(-0600)] <rickard> Sure
[14:17:52 CST(-0600)] <EricDalquist> that way we don't forget to do it
[14:18:09 CST(-0600)] <rickard> While we're on the subject, any news on 3.2.5?
[14:19:11 CST(-0600)] <athena> thanks for tracking that down rickard!
[14:19:36 CST(-0600)] <EricDalquist> I'm working on fixing a portlet preferences bug in 3.2.4 right now
[14:19:46 CST(-0600)] <EricDalquist> and then have a few things from our 3.2.4 impl to merge
[14:19:51 CST(-0600)] <EricDalquist> it is on my list and near the top again
[14:19:58 CST(-0600)] <rickard> Ok, cool.
[14:20:13 CST(-0600)] <EricDalquist> this is my logic map for portlet entity persistence that I realized we need to follow:
[14:20:13 CST(-0600)] <EricDalquist> persistent with prefs & in db - update
[14:20:13 CST(-0600)] <EricDalquist> persistent with prefs & not in db - create new & update
[14:20:13 CST(-0600)] <EricDalquist> persistent with no prefs & in db - delete & create interim
[14:20:13 CST(-0600)] <EricDalquist> persistent with no prefs & not in db - noop
[14:20:13 CST(-0600)] <EricDalquist> interim with prefs & not in db - create new & update, delete interim
[14:20:13 CST(-0600)] <EricDalquist> interim with prefs & in db - get db version & update
[14:20:14 CST(-0600)] <EricDalquist> interim with no prefs & in db - delete db version
[14:20:14 CST(-0600)] <EricDalquist> interim with no prefs & not in db - noop
[14:20:33 CST(-0600)] <EricDalquist> we only have 1/2 of that logic covered right now
[14:20:34 CST(-0600)] <rickard> athena: No problem. Now it's up to you to sort out the admin stuff for supporting multiple locales in the portlet
[14:20:54 CST(-0600)] <EricDalquist> also I think our goal is to eventually just be UTF-8 out of the box
[14:21:03 CST(-0600)] <EricDalquist> so tracking these things down is good
[14:21:11 CST(-0600)] <rickard> Yup
[14:41:05 CST(-0600)] <rickard> EricDalquist: https://issues.jasig.org/browse/UP-2926