uPortal Developers' Meeting
Johns Hopkins University
April 23-27, 2007
[editing has been opened up to attendees for corrections]
Attendees
Michael Oltz -- Cornell
Jonathan Markow -- JA-SIG
Chris Doyle -- Johns Hopkins
Theron Feist -- Johns Hopkins (Tuesday only)
Phil Genuzzi -- Johns Hopkins
Elliott Metsger -- Johns Hopkins
Dwight Raum -- Johns Hopkins
Jason Shao -- Rutgers University (Tuesday only)
Colin Clark -- University of Toronto, FLUID Project
Michelle D'Souza -- University of Toronto, FLUID Project
Peter Kharchenko -- Unicon, Inc.
Andrew Petro -- Unicon, Inc.
Eric Dalquist -- University of Wisconsin - Madison
Jen Bourey -- Yale
Susan Bramhall -- Yale
by video at Boston University
Kim Soo Il -- Boston University (Sakai)
Amir Rashid -- Boston University
Peggy Wang -- Boston University
by video at Rutgers
Faizan Ahmed -- Rutgers
Dan Ellentuck -- Columbia University
NOTE: When it says a session is "by" someone it does not necessarily mean they have been responsible for the work or action item in question. It means only that they presented, discussed, or facilitated that session.
A few terms used elsewhere without explanation
CAS -- Centralized Authentication Service, a single sign-on system originally developed at Yale, but has been contributed to become a JA-SIG project.
Layout manager -- that part of the uPortal infrastructure that assembles a user's layout for display, and allows them to change it.
SLM -- Simple Layout Manager, the earliest model still in use. The user sees a default layout until they make even one tiny change. At that point, a custom copy of the entire layout is made for that user, and the user will see no further changes the site staff make to the default layout.
ALM -- Aggregated Layout Manager, introduced the concept of "fragments", tabs which are maintained by site staff and which are kept up to date in the view of each user.
DLM -- Distributed Layout Manager, has capabilities parallel to ALM but considered an improvement. Contributed by SunGard SCT to the uPortal code base.
MONDAY APRIL 23
Future options for conferences and meetings
by Jonathan Markow
presently 2 conferences a year
June conference is uPortal -- has become more inclusive
December -- broader scope, tools, middleware, technology in general. not as well attended, although the people who attend get a lot out of it.
JA-SIG has been losing money on the conferences, because they used to be sponsored by Sun Microsystems. They have dropped back on the sponsorship. Can't continue to run the conference that way.
Board suggests for December, a different format to lower the conference management costs. What do the people here think:
June -- "JA-SIG One" -- Projects all over the spectrum, including uPortal, fully fledged -- has an extra day now.
December -- "JA-SIG face to face" or "JA-SIG working sessions" -- focus on working out issues on specific projects, etc.
- hosted at an institution rather than a hotel
- less need to manage proposals and registration
- minimal "track leads"
- have activities organized across: e.g. uPortal developers; coding sessions; bar camps; bug fixing marathons; documentation marathons [someone said "they would need extra beer"]; HyperContent; sakai/uportal collaboration; put together grant proposals; board could get together; training; i.e. various self-organizing groups
This would be less costly, would need an overall organizer/program chair; and leads by kind of activity.
Opened the floor for comments.
Eric: really likes the idea; if this happened, we would have only one other developer meeting and maybe some video updates. Good place for people to sit in on and grow their knowledge.
Susan: Would relieve the burden of in-organization organizers; and the more 'sessions' we have, the less networking occurs.
Jonathan: Invite non-JA-SIGgers to come together and work on projects at the same time. e.g. Moodle.
Andrew: Would Concentra not be involved at all? Concentra is adding a lot of value in making the conference happen. I'm concerned about the ability of volunteers to execute on the logistics sufficiently to make a December conference happen without Concentra.
Jonathan: We would use them some but less, to economize.
Susan: "bar camps" just invite everybody together to work together and present. The organizing, you don't want programmers to do that.
Eric: You need someplace for people to stay and places to meet. The less organized it is, the less friendly it is for new people; "what's going on?"
Jonathan: It would be workable for already well-defined groups.
Eric: How many in-person developer meetings per year do we really need?
Eric: I feel more productive sitting down and working with others. Do we need two, and if so when should we have them.
Jonathan: You need at least one pure developers meeting.
Andrew: They are useful milestones for setting goals and measuring progress. There's the issue of momentum.
Jonathan: There should be a steering committee for each project. He discussed what the composition of such a group would be, to keep the momentum going.
Eric: How about two or three scheduled video conferences to check up on progress, and peg it down far enough in advance so that people can hold the day(s) open.
Susan: I think having them more often by video would keep things more on track.
[Rutgers video was not going when we began; they have now arrived and introduced themselves]
uPortal 3 Status Update
by Eric Dalquist
Elliott, David DeWolfe, Eric, and Peter met and changed uPortal 3 over to Pluto 1.1 (a significant change). Got about 25,000 lines smaller because 1.1 has those things built in.
We are moving to Maven for project management. More modularization of the framework as we go forward. Make the API's more independent of each other.
The uPortal 3 in the trunk uses Maven and passes all the tests. You package up an "EAR" file, which is shared libraries at the top level and a set of WARs including uportal proper as one of those. It is possible to put one EAR into Tomcat if you're careful; and it should drop into any J2EE container well.
We have removed the custom portlet deployer from uportal 2 and are using Pluto's own 'assembler' capability. Now using the Spring caching package instead of custom caching.
We're using plug-ins like JSP compilers.
[keep in your mind: Things that uPortal depends on should switch over to using Maven, e.g. GAP.]
Using Maven (2) for uPortal 3
by Eric Dalquist
Eric thinks of Maven as an Ant scripting language. way better than make files.
advantages:
- standard structure for the folders
- don't have to write code as to how to compile, or package
- Maven does auto dependency calculation
- Have to check out less stuff from subversion (fewer jars in there)
Elliott: Getting to understand the terminology is one of the barriers.
Eric: There's a link from agenda on how to build up3 with maven
You have to put a few dependency jars in by hand because the licenses don't allow them to be distributed from any public Maven repository.
And you have to build some parts of uPortal to get them into your local deposity (see that page).
There are several ant utilities that help with this building process, they are listed at the bottom of that page.
Susan: There are aspects of configuring uPortal that you have to run uPortal in order to change it.
[some discussion]
uPortal 3 In-Place Configuration
by Eric Dalquist
Eric: We should be able to state things as XML, drop in some designated directory, and when uPortal starts up it would notice the file and do what it says. There will eventually be no 'dbloader' it will work this way instead.
Eric: It's time for exporting and importing databases or configurations via XML.
Susan: How do you get anything close to a fresh start when you have several years of experience and growth in the database and configurations?
Eric: We need to expose the administrative interfaces as Web Services. Other things should be well-documented API's.
Susan: As long as it's sort of "heroic" to upgrade from one uPortal version to the next, it doesn't look like a serious contender.
Eric/Andrew: Upgrading from uportal 2 to 3 is going to be inherently hard.
Andrew: One of our developers (Drew Wills at Unicon) is addressing migration from one version of uPortal to another. Currently this has been between uPortal 2.x versions, but it lays a groundwork for exporting data from a 2.x instance for import into a 3.x instance.
Eric: It would be simpler to upgrade database is to do it incrementally, one minor dot release at a time, from 3.0 to 3.1, 3.1 to 3.2 etc. Dump database to XML, and run XSLT on it per incremental upgrade. There are streaming XSLT parsers that don't require the target XML to be in memory all at once.
Eric: Many times an upgrade consists of adding new fields rather than changing existing fields; this is not as burdensome.
Susan: U Wisc uses DLM. How do you move a fragment?
Eric: We do it by hand. It doesn't take that long. Keep good lists of what you do each time.
Faizan: The layout publishing tool. We have a 'publayout' tool but it only works with SLM so far. The code came from Ken Weiner specially for Rutgers. They would be willing to share it. It is very handy.
Susan: It should be pretty easy to port that to say DLM.
Andrew: It's my impression that there are still some sites on plain SLM that could
Faizan: I will email code to Andrew. [Faizan has since done so and, predictably, Andrew hasn't had time to do much with this code. It needs to get out into public source control for collaboration, so that others can pick up the torch, so that people motivated by need can run with it. ]
Return to discussing uPortal 3 status
Eric: We did RC1 before the last meeting, there were serious issues. we addressed some of them. There was a release last Friday, "Milestone 4". Still has some bugs so not quite good enough for RC2.
Andrew: And there are some intended new features that aren't in it yet.
Eric: For the summer conference, (tentatively "RC2"), from most likely to least likely by then:
- restructuring the Hibernate DAO's, could be rather simpler; simplifying our registry logic regarding that
- taking what we get out of our XML talks and implementing db import/export so we can get rid of dbloader. domain-specific XML info (GAP, etc.)
- hopefully what people want out of the box. We haven't tested all the different group stores yet.
- how about an integrated LDAP server (via Apache?).
- drop-in configuration of a portal.
The first two Eric feels have to be done, the rest he would like to be done before the conference. -- he wants to cut the release about a week before.
uPortal 3 XML data formats for Import and Export
by Eric Dalquist
Eric has a page of requirements for this feature (summarized as the list below).
It's a pain to have to tinker the configurations and upgrade the database.
- Should not have an independent app to tinker the db. All should go through the standard DAO's.
- Object re-use -- use the existing read and write code
- There should be a migration path between dot releases.
- Should run from command line without running portal, and from the web UI.
- Should have varying granularity, a single object, group of objects or the entire database.
- XML format easily human readable or writable
- Have schemas for all the tables so XML editors will nicely edit them.
- Include an object model version number so XML will read earlier versions.
- A way to reference other objects in the same XML file or other XML files already in the system. This is the hardest question Eric has about how to do the export/import.
- DAO implementation agnostic. -- don't include database specific keys in the export, and shold not be needed in the input XML.
- Functionality should work for all portal domain objects including: users, portlet domain objects (6 different objects), groups and layouts.
[morning break]
Eric: Export/import isn't so much "actions" (move channel from group a to group b) but more a different representation of the data. Sort of object serialization.
Elliott: It's exactly object serialization.
Andrew: There's that field in the persondirs table that's misspelled.
Susan: What is this export/import for?
Andrew: An incremental unit of change.
One way to look at this -- to make a change, you export, change the exported thing, and import it again.
Peter: We've had some of these things appear and disappear over time.
-- there will be things like the object structures, very streamlined, XML or one line mappings.
Peter: Do this implementation through existing (DAO) API's. If you look at those with respect to DLM, you'll see that there are some things missing.
Eric: What do we need to include?
- layout information; depends on what LM you are using; primarily the user's layout.
- layout fragments; kind of the administrative kind of layouts.
- objects: there are some things that are channel definitions, but there are several that are not channels. how does this work if you only want to export a few of them.
- portlet group object
- local group information
- local permission information
- user account information
- Group and Permission information.
(Dan: there are bunches of things that are not handled by the portal framework.) (Data in the local store may reference things in other stores; especially the permissions table) (Eric: exporting the PAGS group information isn't similar to other tables; it's rules rather than users or channels) (Dan: there's an implied plan to persist the PAGS rules in the database, so you have to serialize them as well as the explicit memberships in the local group store) (Eric: having an XML schema that fits into the export/import scheme would make the exporting and importing just work)
Eric: We need a distinction between restore versus publish.
Susan: We need a way to 'remove'
Eric: Overwrite versus add.
Andrew: Maybe we can have a flag to make it explicit -- if there's something in the destination database that's not in the input, toss it out.
Eric: How do we tell the tool what to do? Is that part of the XML document? Or is it a command-line option
Susan: Would we have a different set of XML schemas to delete with?
Eric: "Add," "Update," or "Remove" would be options to the utility.
What do we do if we're asked to delete a row that isn't already present anyhow?
If we delete something, should we only set a timestamp to say it was deleted, or shold we actually remove the record?
When we delete a record, we will have to delete other records that refer to it.
Eric: In the first pass we should only "mark as deleted" instead of really removing it.
Do we need an option in the UI of to "sort of delete" or "really delete".
Dan: I've found it convenient to do deletions the current way, but it confuses novices that the channel isn't removed from the db.
Andrew: Needs better UI warning message for that.
There are features in up2 for a channel to become active at a specific time.
Eric: This will probably just be a boolean for up3, not formally timed.
Eric: How do we deal with referencing objects? We now have a whole domain object hierarchy instead of a simple "channel" thing. Harder to reference into those.
Elliott: If we want to be able to hand-edit the XML files, this is a problem. For exmaple Confluence and Jira tell you never to hand-edit.
Eric: Is there some data that you would only need when exporting from one portal and into another portal, that you wouldn't need if you are updating or maintaining one portal?
Eric: To introduce more granularity we will have to increment the XML schema version numbers.
Eric: Define a schema for each object you want to define. A portlet schema, a definition schema, a layout schema. Could reference things by fname, or you could reference by container. ("The X which is in Y") But that last one seems to be scary the more I think of it.
Andrew: An export which exports the domain model as it really exists, and the "make life easier for Andrew" tool that does that but only to the point that it gets painful and does some tricks to be more human readable and editable.
Eric: So we have the use case of migration (export from one database, import into another)
Susan: Perhaps we should discourage hand-editing.
Elliott: Yes, there should be tools instead.
Eric: If we drop that requirement, then the JDK XML serializer/deserializer would be helpful. We are going to write some tool, but if we don't care what it looks like externally, pre-existing code is useful.
Eric: We are not suggesting that these XML pieces are for the purpose of being edited; they are for temporarily storing info outside of the database.
Eric: I can see someone exporting, importing into a scratch portal instance, editing it via the portal UI there, export and import it back to where you started, in order to do a moderately complicated change to the portal.
Susan: So we need a layout manager that lets the portal manager edit the layout. Minimally, edit fragments, edit user layouts.
Eric: We don't want to make this XML easy for a person to deal with. Rather "if there's something we need to do in the XML right now, the approach should be to fix the tools so we don't have to do it in the XML."
[LUNCH BREAK]
Peter: issues: a) serialization of beans to XML b) coordination -- all the various pieces working together.
Elliott: The things that do the serialization and the things that do the coordination will be different things.
Peter: We could put together two different projects to do that.
What about "Xstream"?
Elliott: I've used that and it's really nice -- it is a low level thing and can use just about any parser you want. The programmer has detailed control over how the process goes. You can provide a "converter object" to handle things that are not built in to XML.
Peter: Does it handle collections OK?
Elliott: Yes.
[how does it handle relationships?]
Eric: It uses a form of XPath for relationships.
uPortal 2 XML import and export
by Andrew Petro and Andrew ("Drew") Wills also of Unicon Inc.
Powerpoint and code sample available as attachments to the wiki page for this meeting session
The session for tomorrow on XML import/export for uportal 2 using Cernunnos, was moved from Tuesday to Monday, as it is closely related to what we were discussing before lunch.
presented by Andrew Petro in person, and Andrew ("Drew") Wills who is actually doing the work was present via speakerphone to make clarifications when needed.
Andrew: There is the mechanics of (importing and exporting data) where the readability is not so important, and there are the administrative tasks which humans have to be able to do.
Problems with moving and editing uPortal data.
One solution: write out data to XML. But the "domain specific language" does not make it easy to understand and change the data.
Parent-child relationships can be represented naturally in XML with no extra work (parent encloses child).
Drew went out of his way to avoid using reference ID's.
Andrew showed an example of representing a layout as XML. (looks like uPortal 1)
Drew is representing most stuff except "users" and various minor things.
He uses Cernunnos which he made, and which lives in Google Code. "provides flexible Java implementations for common behaviors and allows them to be used with a common XML syntax".
An example script: recursively searches an archive for jar files which match a particular string.
A prototype importer/exporter is available which works on 2.5.x and later.
This tool does not represent groups themselves nested inside one another. Each group gets its own document. Then membership nodes or relationships each get their own document. So if a group contains 7 members there will be 7 very short documents, one for each. If a group is in 3 different other groups there will be 3 documents representing those memberships.
Dan: What are the pros and cons of references where a reference is replicated? How have you conceptually dealt with that issue? [i.e. the discussion pointed out the natural nesting capability of XML, but it doesn't look like this uses this much.]
Dan: Does this mean that there are a whole lot of new pointers or identifiers that are created just for this purpose of linking?
Drew: No, I use existing identifiers such as USER_NAME which are supposed to be unique. The critical thing was to get away from database ID's (auto generated integers for example).
Andrew: This is concrete code. If we really like this we could make it part of uPortal and have it as 'the' solution of this specific problem.
Eric: How do you feel about the project? Do you plan on continuing development?
Drew: I would certainly finish it through the "users" part and use it with the professional projects I'm involved with. Is there anyone else who would want to use this and try it out, so Drew could get a separate perspective on it? And wants to hear from other people whether they think it would be of use to them? I see all kinds of opportunities to do some little projects to fix an issue, so I wrote this to make those feasible in the time that I have.
Susan: I think it looks extremely useful; I wish I had known about it six months ago.
Chris: Looks good.
Susan: It looks like there's a learning curve. But it will probably be worthwhile.
Eric: Why don't you, Drew, send out an initial note on uportal-dev list telling what you have and see what happens. I can put you in touch with someone at U Wisc.
Dan: It would be a win as a migration tool. What would be the scope of work in making such a thing?
Drew: I have lots of thoughts about that. Each XML structure or document in the root references a script in the classpath. Just put a script there that migrates the data forward and then writes it out.
Faizan: If you want to use it to convert from SLM to DLM, we could try that here at Rutgers.
[Andrew is on the projector, so Eric asked him to just continue with a later session]
Recent Unicon OpenSource Activity relative to uPortal
by Andrew Petro
Powerpoint and info in wiki page representing this session in Confluence.
San Joaquin Delta College
-- Columbia Announcements further customizations to work better with Rutgers Alerts use cases. [i.e. add notifications to it]
-- Email preview portlet.
work on 2.6
-- release engineering activities
-- fixes and enhancements from Academus merged into uPortal.
"OpenToro"
Academus 2.1 was codenamed "Toro"
The functionality that is in 2.1 is being opensourced [little details that are not]
They are not opensourcing the brand name.
toro-channels
-- IChannels and frameworky stuff
toro-portlets
-- portlets
toro-sdk and toro-resource-pool (predated common-dbcp)
-- utilities
http://code.google.com/p/toro-channels/ [et cetera]
toro-channels:
- classifieds
- news
- collaborative groupware course management system
- survey
- calendar
- bookmarks
- gradebook
- LDAP password change
- notepad
- notification
- permissions admin
"Eve" performance-enhanced local group store
"CSCR" client-side channel rendering technology -- asynchronous channel rendering using frames. not the most modern take on what that would look like.
toro-portlets
- gateway Single Sign-on
- briefcase
- messaging/notifications
- permissions management
toro-resource-pool
-- predates Commons DBC
-- nice debugging/logging facilities
Why open source?
Easier to deliver on support and services on a platform everyone can use "eyes wide open"
Going out under GPL
What is Unicon's angle?
Healthier JA-SIG projects lead to more adoption which leads to wider customer base
Now that it's open source, others may fix bugs, answer questions
Why Google Code as the repository?
- Path of least resistance
- Cheap easy and fun
- SVN source control
- Issue tracker
- Wiki
- Downloads
- Online account management
Andrew reviewed the various channels
Announcements -- further enhancement of the USC enhancements to Columbia
Email preview channel -- currently replays the password, will CASify this. Peter has got involved a bit. -- Andreas Christoforides, and Mark McClellan.
Calendar -- can import and export but can't just point at external iCal.
Poll
Client-side channel rendering -- bad for screen-reader users, so it can be turned off on a per-user basis.
... and the portlets
Notifications portlet
Briefcase portlet -- doesn't support WebDav yet
Gateway Single Sign-On --
[Eric at U Wisc has a beatiful Bookmarks Portlet]
We don't want competing bookmarks project.
This is under GPL "with FLOSS exception" -- and so uPortal can use it anyway.
The Gateway SSO is store-and-foreward; it's only for people who are not using existing SSO solution.
Calendar channel: Bedework, Zimbra, Chandler, why this one?
Faizan: the iCal rendering channel, Jason was originally going to be releasing that. But Faizan saw that there are some error handling problems, so he pulled it back from the release. (Jason is going over to Sakai team at Rutgers, from the portal team)
Briefcase -- could we merge it with the Rutgers one?
As an aside: On "Projectness"
Some uPortal channels and portlets outside of the main project don't feel very "projecty". They lack one or more of:
- discussion lists?
- releases?
- issue tracker
- progress?
Likely untapped value in getting more "projecty"; would facilitate functionality merge
It would be better for these channels if they either get more attention as independent components, or get sucked into the core release. uPortal is the only platform in which you're going to use an IChannel, so how about just ship the channel with the platform and you can use it or not but you can skip the pain of figuring out how to install it.
Jonathan: Well but we've thought of the portal itself as a content-neutral platform. One of the directions for JA-SIG is to put ourselves in a position of incubating new projects.
Susan: Why don't you put all these channels into one "channel project"?
Jonathan: Wouldn't that be more difficult to administer? We could use a more robust Clearinghouse with more features. Aaron Godert is looking into this; Google Code is one of the options being considered.
[afternoon break]
UW Email List Hosting Transition
by Eric Dalquist
This is now in progress. We will have 'uportal-dev' and 'uportal-users' and it will be easier to have specific mailing lists. It's off "uportal.org".
Hypercontent access to the uPortal Developers' site
by Eric Dalquist
It is suggested that we let people other than John Fereira update the site.
[developer.ja-sig.org] Presently you write email to webmaster@ja-sig.org.
Jonathan: I think we should have a couple more people doing this, but not a lot of people.
GAP, Maven, and Hibernate
by Eric Dalquist
Dan: [some questions related to GAP]
One work item is: fix some integration problems between uPortal and PAGS.
Eric: The Apache Directory project -- an embeddable LDAP server. Wanted to move some things into there and have PAGS work on it so people could try this out on uPortal.
Dan: I think that's already working, but we need to encounter some real life problems to be more confident in it.
Dan: Concerning the switch to the Spring caching API. GAP has its own caching API. It has always had different requirements. Is Spring distributed?
Eric: Spring is actually a facade. Can plug in EHCache or JCache or OSCache etc. Uportal 3 has completely gone away from the idea of user-specific caching. For example, there is a portlet window cache. Maintaining a _separate_ cache per user is too much overhead. Instead there is one cache per object type [which is indexed per user when appropriate].
Dan: Have you been using e.g. OSCache through the API with distributed caching, or is this more of an opportunity for people to try it out?
Eric: We're using EHCache right now, no particular reason. I might get to it eventually but that might be as much as a year.
Andrew: The Academus product documentation is not yet available with the toro stuff. We're going to have to 'airbrush out' the trademark or something. Until then you're on the bleeding edge.
Faizan: asked about the Email Preview channel [apparently he was out of the other room when this was discussed]
Andrew: briefly covered it again. It reasonably handles errors. It will handle only one account, that of the person logged into the portal; it can't switch among multiple email accouts used by that one user. But it's pretty pluggable and modularly designed - it's pretty clear how you would extend it to do all these sorts of things.
Jonathan: ECL and Apache have slightly different licenses, and JA-SIG wants to change toward an Apache 2.0 or 2.1 style license. We want to be on an OSI-approved license.
There was some discussion re licensing. For example, we are not eligible for Google Summer of Code funding.
Andrew: Where did we get our license [the one used on uPortal]?
Jonathan: Wasn't that the MIT license?
Andrew: It looks something like that, but it looks more like the Apache 1.1 license.
Tuesday, April 24
uPortal 2.6 Status Update
by Andrew Petro
Drag and Drop user preferences
- turned off by default
- your theme needs to be upgraded to use it
Pluggable DLM processing pipeline -- in dlmContext.xml [in properties dir]
Now that we have a "hammer" (DLM injection)
- Layout restrictions
- Alerts-like functionality
- Just-in-time insertion of "stuff" into the layout
-- Security advisory around RSS reader instances?
On Platform Maturity:
uP2 has warts, but also has "platform maturity"
- existing features
- known limitations
- installed base
- experienced users
how do you add value
- fix nagging issues that sap value
- add features pointedly to increase value
- add documentation
DLM is the default in 2.6
Jonathan: We need to prepare people well for this release, because drag-and-drop is going to get a lot of attention.
back to Andrew: Jason has been working on documentation. It's on the wiki. Not quite ready for release. [Since the meeting this has been opened up for public comment and participation as the "uPortal Manual" wiki space, but not yet pushed hard out to the world because it still has large gaps.]
On release availability:
- There is a 2.6.0 wiki page.
- You can download RC1
- Vincent Mathieu made a quickstart [has been posted since the meeting]
There is a CSqlQuery channel. Could it be subject to SQL injection attacks?
What about JChart?
Skinning 2.6:
Customizing the XSLT is optional
Oh, about those XSLT's, there is now a JSP channel
JSP as alternative to XSLT
Channel -> XML -> XSLT -> HTML
Channel -> Model -> JSP -> HTML
CSyndFeedReader
Backward compatible with CGenericXSLT
Based on Rome library
XSLT -> Java processing (more robust)
End-user-subscribe-time configuration
-- Jen added runtime end-user config
Fixes from the 2.5.x. series should also all be in 2.6.0 M1
Requires JDK 1.5 or 1.6
Tomcat 5.something
Refactored channels away from IMultitheaded*
- Reduced lock thrash
- Simpler
- Easier to debug/log
uPortal CAS integration convergence
by Andrew Petro
The CAS support available to work with uPortal has been evolving.
Added the CAS support into uPortal codebase as of 2.5.x (.2?)
There eventually resulted two competing attempts to define shared CAS integration APIs
Should we retire the APIs in uP and standardize on those in JJCC? or should we retire the APIs in JJCC and standardize on those in uP?
Scott from Rutgers was going to participate remotely as he made one of the competing APIs. But an emergency came up and he could not spare the time.
The current model for authentication is the login and the PersonDir.
Jason: First question, where should these API's live? In the uPortal codebase or outside? Jason thinks, inside. Use a useful level of abstraction so that it can cover CAS and other things.
How many LDAP security contexts do we need to maintain at present? One for uPortal and one for (the separate) CAS (distribution).
What embedding CAS does for us is that it brings in a publicly documented API for doing auth, rather than having something idiosyncratic.
Andrew: So how do we resolve this issue?
Jason: Let's produce a distribution with uPortal and CAS, and a config that lets our existing authentications talk through CAS.
[pop the stack back to talking about uPortal 2.6]
CGenericXSLT System Cache Scope
ability to cache CGenericXSLT output across the portal [instead of merely user by user]. this is configurable per channel.
There is a "Milestone 1"
Future for 2.6.0 towards release:
- A little QA
- A few more fixes
- An RC
- A GA release in May, ideally soon after the dev meeting
Jason: We have a document caching implementation; we cache the document as it comes in, instead of after the XSLT. We cache the XML document; no params are passed in; we piggyback on the ICacheable implementation. This way there can still be per-user customization of it (e.g. show only part of it) while still getting the benefits of caching.
Let's try to put the document caching on 2.7.
[morning break]
uPortal 2.7
by Andrew Petro
Jan Nielsen's (from SunGard SCT) vision for platform maturity
Better platform for building something bigger
- more formal identification of API's, more care in changes to API's
- Better platform for a commercial platform built on uPortal
- Better platform for a campus portal built on uPortal
Here are some suggestions for what to add for 2.7
Mark Boyd and DLM evolution
-- he put some enhancements in HEAD
-- There are more features available in the Sandbox
-- Let's merge these into the core, QA, and release uPortal 2.7 with better DLM
-- (more on this in a later meeting slot)
CAS server integration
-- Proxy CAS for the masses
-- Easier -> better documentation
Rutgers remote document caching
-- Can this make 2.6?
-- Or does it need to go into a 2.7?
Portlet support
-- Move to Pluto 1.1 (uP3 went to this)
-- JSR-286? The revised, not yet final, portlet specification [not included in Pluto 1.1 but doing the Pluto thing positions us for 286]
-- Fix for the dreaded UP-1040 bug [a multiply-published portlet can get preference settings confused between instances]
Jonathan: Chuck Severance suggested that David DeWolf should put WSRP support into Pluto 1.2.
Andrew: WSRP is nice to have and allows you integration with various things, but we have other things that are more important.
Jonathan: There are a subset of large institutions who see a value in WSRP support in the portal. They are not here today. We should have at least WSRP consumer.
Eric: Writing a WSRP producer is a significant amount of work.
Creeping Featuritis
uPortal gets dinged in side-by-side comparisons with other portals on less out of the box functionality (portlets, channels)
Energy is lost to "friction"
-- There are a lot of neat opensource plugins for uportal
-- It requires lots of calories to figure out how to get them all working in your portal
code versioning friction
-- for example, what versions of the announcements channel will work in what versions of uPortal with what modifications?
-- merging source into uPortal source tree allows each uPortal release to be developed and maintained as a cohesive whole.
The uPortal channels CVS is not used as a working repository but as a stick-it-in-there repository.
Energy is lost to installation friction
-- Even when the code will work perfectly (no version mismatch) it's still problematic
-- find it, install it, maybe provision a db, configure/publish the channel, all just to try it out.
We could put the channel source into the main tree, and have them built at the same time, or have separate build scripts for each channel. In either case we will be better able to see what compilation problems there are.
Our target for uP2 is not, let's make lots of new IChannels, but, what can we do with what we've already got.
Jonathan: If we integrate these channels, would that make the expectation that the same things will be available in uP3 as portlets? [This would reduce interest in uP3 if they are not included]
Jason: Who owns this vision and who will drive this forward? What portions of this idea are discouraging to viability of 2.6?
DLM XHTML/CSS skins for 2.6
by Jason Shao
2.6 includes a somewhat genericized version of the Rutgers theme, which uses XHTML and CSS.
This uses the XHTML serializer so it is more demanding on input.
Rutgers has done an internal refactoring of this skin.
Now the other skins are just metaphorically "extends" of the primary CSS. So the main one is 450 or 500 lines, and the other ones are about 70 lines.
Some info about the new "widgets" they're working on. Consolidate the internal presentation of their specific tables. Toolbars, tabbed panes, tabular data. They are trying to standardize not only within uPortal but on their other web sites as well. This effort -- they have some Photoshop comps reflecting their thinking, they have no expected date for completion.
We have new skins -- such as pink or pink fall skin.
Drag and Drop for uPortal 2.6
by Jen Bourey
Most features copied from the preferences channel
Browser compatibility: IE6, IE7, Firefox, Safari [not testing on Linux, Opera or Netscape]
Fails back if scripting is disabled
Javascript Resources
Uses the Dojo! AJAX library
Portal dojo classes
Implemented in ajax-preferences.js
Servlets
-- Channel registry
-- Ajax callback targets
Theme param injector (turns the functionality on)
Skin resources
{$skin}_preferences.css
Extra icons
Channel editing capabilities:
- move
- delete
- min/max
- adding channels
- browse channels by category [channels which were nested using previous preferences editor, are shown in the correct category, but the nested structure is not visible, it appears as one level.]
- search channels
- can use channel without adding to layout
- add to my layout
- put it in some other tab
editing Tabs also work ajaxy.
Susan: The "use without adding" makes the concept of a layout less important to me. It's more, "find something to use and use it".
Andrew: The idea of a dashboard ("tell me what's going on at the moment") means that a layout is still useful.
UI Testing results
- Confusing terminology
- Channel adding should be more obvious
- Drag and drop tabs?
- Simplicity
- Users want less tabs
- Possibly too many features
Jason: "widgets","modules", and "gadgets" are more the current nomenclature.
Jason: Should we distinguish to our users in terminology between stuff provided by the site staff, and things the users put in themselves (RSS, photo pages, etc.).
Colin: FLUID could put some time into working on the terminology.
Jason: Are you going to fall back to the regular preferences channel for advanced config?
Susan: No, probably not; it requires too much support.
Jonathan: Can users move channels between tabs?
Jen: Not yet. Not for 2.6.
We won't have the concept of setting the active tab to any other than the first one. Just move it to be the first one.
Jason: Does Yale have DLM-locked content?
Susan: I think we'll have to lock the tabs, because we would have no way to push new content.
Unimplemented features
- skin selection
- resetting a layout
- setting the active tab
- moving columns
- moving channels to a new tab
- Asynchronous channel rendering
Showed examples.
Such as, integration with Google Appliance.
A short discussion about issues using Ajaxy Javascript, e.g. preventing namespace collisions, some scripts aren't designed to be included more than once in a page, using embedded IFrames.
[lunch break]
Asynchronous channel rendering
by Jen Bourey
[no formal presentation, just discussion] in other words, redraw one channel without refreshing the whole page
Unicon has done some stuff with frames. Shawn Lonas (shawn@unicon.net) and Gary Thompson (gary@unicon.net). Academus: there is a "download worker" a channel can give you a link to download something produced by the portal. So they put in a callback and the target of the callback is the content of the channel.
The problem of links within such a channel was pointed out: how do you intercept when the user clicks on it, so it can stay inside the window. Especially with respect to a portlet.
Eric: Any URL that a portlet generates has to go through the API some way before it comes out.
There was some discussion of this issue.
Andrew: Perhaps better, if we have as our wish, that all channels should refresh asynchronously, and we have some kind of markup or parameter to indicate which ones would not work that way.
Peter: That would require implementing channels both way and supporting that.
Jason: So is the next step to make a working group and set up a conference call or something?
Colin: Melissa and I would be interested in being involved in that discussion.
Followup on DLM Status
by Andrew Petro
Mark Boyd was at SunGard SCT, where he contributed much work to the Distributed Layout Manager and moving it into the uPortal code base, but now he has moved on to other things.
DLM: he has some material contributed to 2.6.
There are more features in the Sandbox. The main reason is that it was dependent on a robust internationalization implementation that is not in the base. That code particularly addressed the fragment management editor.
Mark could probably answer questions but would not be able to work on code.
Who owns the DLM now?
Jonathan: Who has relevant expertise?
No one present was able to commit to pick up the role.
Susan: What do we mean by 'owner'?
Jason: That would be up to said person.
Andrew: Maybe the question 'owner' is too strong. What should be our response to this?
Susan: There is the question of completing Mark's current material for uP2, and the question of having it in uP3.
Jason: Pulling this out to a shared module is not a reasonable option.
Susan: It would be unreasonable to release DLM as the default (in 2.6) and then abandon it (which is the alternative to doing something about DLM).
Eric: There is more work involved in managing a portlet under 3.x, so the code will indeed be quite dfferent. The functionality will be the same and that will seriously inform the code, so it's not completely in a vacuum.
Jason: For 2.x, our posture is apparently that there is no formal owner of DLM at this time, but there is community interest and support, and for specific issues commercial help can be hired.
Andrew: Apparently the decision is, we coast for several months and see what happens, and do not as yet try to do anything about the Sandbox.
JSR-168 support in uPortal2; status and prognosis
by Andrew Petro
There are a few sites present running portlets, and some who want to.
There was a time where JSR-168 was a compelling idea but there were very few portlets worth running. That is by far no longer true, there are lots of interesting portlets now, and there are interesting frameworks that support it.
[Andrew and Susan are going to be representing channels at the "Channels vs. Portlets Smackdown" at the summer conference]
a) UP-1040 -- a Jira issue about mapping channel parameters into portlets. Should be able to multiply publish a single portlet. There is a big bug where if you do that they randomly see each other's parameters, etc. Eric says this has something to do with Pluto 1.0, Pluto 1.1 addresses it. Right now UWisc, for example, uses many duplicate portlets with different names-actually the same portlet-so they don't get their parms confused. There is now a patch that Tim Carroll wrote that may fix it.
b) Pluto 1.1. JSR-168 support in uP2 consists mostly of CPortletAdapter and Pluto. So what we need to do is drop in the 1.1 jar and rewrite the CPortletAdapter. How much work is it?
Eric: Ken Weiner tried to sandbox it -- a channel that does portlets. But really there are things that a portlet wants to do that are outside of what a channel can do. So there are hooks elsewhere in uP2 to support the portlet channel. It would not be that simple to alter all this.
Is there anyone sufficiently annoyed by the current state of JSR-168 support in uP2 to be willing to do something about it, or do we coast?
Andrew: One option is to get several schools who can't support two weeks each, together to hire somebody to actually do this.
Susan: Asking here isn't the end of the question. Many interested parties may be elsewhere.
Eric: UWisc did a workaround to sidestep the problem and that's all we can contribute at present.
Jonathan: Can we broaden where we look for help to do this? Posting to the mailing list seems to have worked before. Is there more than one bullet point for which we could ask for help? Who should get these bullet points lined up and then post a request on the mailing list?
Andrew: There is a 'bite-sized issue' list on the wiki but it has not attracted attention. This sounds like the kind of thing the uPortal steering committee would handle. Is it time yet?
Jonathan: The steering committee is further in the future.
Elliott: Make the ROI clear for each feature.
Susan: A problem in getting help from this group is that the amount of effort that's going to go into uP3 uses up the availability of these people. Could we bring this up at the June meeting where there are more people focused only on uP2 at present? Ask for help keeping the legacy code going forward for a year or two more.
Eric: Get out of this, the list of things that work is needed on. And hand that list off to the steering committee to bring to the community.
Jason: Are the developers the right people to be asking to allocate help? Shouldn't we be asking the project managers or assistant directors?
Susan: That would be the steering committee.
Andrew: I don't think we can wait.
Jason: Right now we're getting lots of requests for statistics, more than development.
Elliott: A couple approaches: (a) See if Tim's patch re UP-1040 will work. (b) delete the Pluto 1.0.1 jar and find the red wavy lines in Eclipse
Mavenizing uPortal-related projects
by Eric Dalquist
this was touched on yesterday.
Eric now has a page on the wiki about how to do this "Using Maven for your JA-SIG project"
For example, you can attach skin and style information to the Maven-generated web site so Eric created a standard for what it looks like.
Jason: Do we have an example portlet or channel that is built on Maven? This would make adoption easier.
Elliott: We have five portlets that use Maven.
[ afternoon break ]
uPortal and the FLUID Project
by Colin Clark
Wiki page for this presentation
Powerpoint slides for this presentation
[During this session I typed in a lot more things from the slides than I had for other sessions. Because the FLUID Project is new to uPortal I decided to leave in a lot of that. But you can refer to the slides as well. There is some discussion here also, especially toward the end. -- Michael Oltz]
Poor usability and accessibility are a signicant barrier for community source
Now is the time to address the concerns of our communities
Our goal is to incrementally improve the overall user experience of Sakai, uPortal, Kuali Student, and Moodle.
This is a cross-project collaboration, a partnership among several universities and corporations
Toronto, UBC, UC Berkeley, York, Cambridge, and others
IBM, Sun and Mozilla Foundation [corporations are providing hardware and cash not their seats]
Broad range of experience
The goals are
Social: Build a community aroud UX [= User eXperience]
Technical: new UI development tools
The process is
- Start with a heuristic and usability reviews
- Establish a baseline for usabillity and accessibility
- Know where we need to improve
- Prioritize the pain points
- Designer's Toolkit: shared design resources
- Personas and profiles
- Scenarios
- UI Design patterns (presenter considers this the best book: Jennifer Tidwell, Designing Interfaces: Patterns for Effective Interaction Design from O'Reilly, 2005.)
- U-Camps
- Iterative UI design and testing
Colin showed and spoke about a flowchart "Component Lifecycle" the process that will be taken with respect to each platform.
Presentation Technologies:
The last thing community source needs is yet another presentation technology
To be different, the technology needs to be fundamentally driven by user needs
FLUID will integrate with the best existing technologies
What are we going to build?
A library of flexible UI components that can be used across applications
A new component framework built specifically to improve usability
Easy to wire up new components or customize properties of existing ones
Components are more than widgets
What does FLUID mean in a Portal environment?
Components capture recurring patterns
Reuse: UI layer consistency
Can be reused across differrent portlets
Build FLUID into cross-cutting areas such as portal navigation
Personalization: UI customizability for institutions and individuals
Focus on uP3: JSR-168 for uP2 support?
A bit about the technology
Unique challenge: how to enable support for very diverse presentation technologies?
Based on JavaScript, DHTML, and AJAX
Thin binding layer between client and RESTful, largely stateless server
Also advanced customization based on user preferences
FLUID Accessibility
Colin believes that AJAX will be accessible; it's just a matter of time
ARIA standard: Accessible Rich Intnernet Application (W3C)
AccessForAll for component metadata
Ongoing tooklit accessibility support
Design specific alternatives
FLUID: Accessibility from the ground up
Ideas for components that FLUID could implement
Resource organization
-- Drag and drop
-- Folders and hierarchies
-- reordering and rearranging
Navigation
-- breadcrumbs
-- paging
-- affordances for external content
Composite, domain-specific:
-- Users, groups, permissions
-- Calendar
Add your suggestions here
Flexibility and Customization
FLUID wil be a highly flexible UI layer
At configuration-time:
-- Appearance, branding, style, page text
-- Locale, reading level, density
-- Functionality and user experience
At run-time:
-- Swap in accessible controls
-- Re-styling for higher contrast,etc.
-- Components built for specific disciplines or user needs
The next slide was a diagram of what the pieces of Configuration-time flexibility would be and their relationship to each other
Another diagram of how they would be used to transform [in the software development sense of the word] the user interface
Core Architecture
- Component framework
- Repository of shared components
- Semantics and specifications
- Integration
Component framework
- Component model and APIs
- JavaScript, CSS, HTML
- Component container
- JavaScript, AJAX toolkit integration
- Runtime Transformation Engine
- Finds and aggregates alternatives
- Server-side binding layer
- REST-based specificatoin + implementation
- REST-based specificatoin + implementation
Integration of FLUID components with any particular project
- Early and often
- Heuristics to measure improvements
- Project integration as soon as possible
- Requires regular collaboration with the developers of the various projects
- Litmus test of project usefulness
Criteria for Initial Components
- Align with project needs/priorities
- Simple, known design
- Complex technically [therefore worth doing, and robust test cases]
- Test AJAX accessibility
- Generalizable
- Addresses "pain point"
- Accessible alternatives
- Learn early!
Jonathan: How will you prioritize the "pain points"?
Colin: We will rely on interaction with the design team, then ask the respective user community what it thinks.
Milestones
- Choose Technology frameworks: May
- Evaluate technology in practice
- Develop real componenets with candidate technologies
- Create prototype image gallery components
- Design, develop, integrate, test, iterate
- Create accessible alternatives or equivalents
- Find other components to get started with
- Aim for a demo at the June conferences of JA-SIG and Sakai
First component: Drag and Drop
- Image Gallery: a mini iPhoto for the Web
- Currently a simple tool written in RSF
- Some clear UX problems to solve
- No way to re-order or sort images in albums
- Plans
- Build components for reorganizing images
- Move to JSR-168
- Create accessible equivalents
- Test in Sakai and uPortal
DnD Accessibility
- What does accessibility mean here?
- Keyboard access
- Low vision
- Single switch/on-screen keyboard
- Focus on the goal, not the task
- Re-ordering images
- Doesn't necessarily look like DnD
- What alternatives are available on the desktop?
- Cut and paste, etc.
- We're starting on research and testing to find out the best alternative solutions
Ideas for uPortal Starting Points
- Jen's drag and drop preferences?
- Help with interaction desgin & testing
- Imrpovements to tab editing
- Refactor to FLUID components
- Accessibility support & alternatives
- uP3 framework?
- Administrative tools
- Other interfaces Eric & co are working on
Susan: Let's get a near-term decision for the non-JavaScript enabled. Which of the currently bad alternatives is the most reasonable?
Colin: My long-term opinion is that we shouldn't concentrate on supporting browsers without JavaScript. Section 508 and the W3C requirements say we should do that, but Colin thinks that these clauses will be stricken as there were technical reasons for them that no longer apply.
Jonathan: I think we need attention to UX of administrative tools, which are poor in that regard.
Eric: A portlet version of the GAP permissions manager should port fairly well to uP2. The API's are almost exactly the same.
Jason: But what about in servant mode?
Eric: Probably would not be difficult to solve.
Jason: It looks like we're going to have stateful components on the client side, in which case reinitializing the session by refreshing the page would mess up things.
Roy Fielding's dissertation on the architecture of the web was mentioned; that's where he coined the term "REST".
Summary
Design and development work is ramping up
Goal: incremental, achievable improvements
We're here to help with your work
Join our community, we need your input!
Qyestions and Discussion
Ideas for starting points in uPortal
individual portlets
uP3 framework
Challenges of AJAX in portlets
AJAX toolkit preferences?
Architectural suggestions
Jen: It would be tough right now to add Drag and Drop preferences to another theme, if this makes it easier that would be great.
Review of last two days and expectations of work between now and the conference.
by Eric Dalquist
[note-taker missed a couple points at beginning]
Functional integration testing.
[Eric suggested let's go around the room]
Elliott: My interest is in working on uPortal 3
Chris: Consolidating efforts with existing portlets and channels and making it make sense.
Theron Feist: still getting up to speed, interested in the UI enhancements. Getting more portlets out of the box.
Andrew: on the hook to prepare for a couple presentations. I expect there will be a build of 2.6 with some name. We have a Professional Services Group at Unicon. Andrew is in the development group, and he expects that there will continue to be development work from them. The developer's manual idea from Jason is a great idea, it deserves some work.
Jonathan: Talk to Chris Coppola about the licensing issues. Work with Andrew to prepare announcements and marketing stuff for 2.6 release. The manual is a very high priority deliverable. Working on various conference preparation activities. Planning around formation of project steering committee. etc.
Jen: I'd like to get the theme stuff committed and clean up the remaining bugs in the Ajax preferences.
Jason: I won't be able to do much in the near future but will clean up the skin enhancements to share, and probably some work on the manual.
Jonathan: Thanks to Jason for working on the two newsletter issues so far (and one more next week). But he's going on to something else. We need a new editor.
Susan: Should be someone who's not a developer.
Michelle: Get more involved with uPortal community, have some Fluid components to demo.
Peter: Fix bugs in uP3, get some more functionality tested.
Jonathan: It's important to put out some kind of announcement about uPortal status and what people can expect by the conference.
Michael: Finishing minutes.
Susan: Focus on upgrade tools. Make uPortal developer forums or a bar-camp that really happens at the conference and is fruitful. And we should concentrate on finishing the skin.
Dan: The highest priority I have is Mavenizing the GAP build. I do want to support any work on upgrade tools that involves the GAP, and efforts to integrate PAGS in real-world settings. As kind of a stretch, I'd like to do some deployer documentation.
Jason: Is uP2 HEAD on the separate GAP code or on the internal code?
Dan: It's on the built-in code.
Jason: Is it doable to be separated out?
Dan: It's doable and I did some work on that, so that the separate GAP has a very similar API to the built-in version. Perhaps after the conference. Very plausible.
Jason: Is that a lot of work or can we solicit someone else to do it?
Dan: I don't think it's a lot of work, there may be a couple dozen integration points in the framework code. The problem is in the custom code, such as in the various Announcements channels that use GAP. We also have to ask how important is it to make this more detached?
Andrew: Part of the value added is it exercises a new version of a library.
Jason: Making uP2 more like uP3 in this way, makes the eventual migration easier.
Dan: GAP depends on PersonDir (because of PAGS), PersonDir does not depend on GAP.
Jason: Removing the "deprecated" messages would be a straightforward task to do.
Jonathan: We should have a bar-camp session on Wednesday afternoon ("BOF sessions").