September 2005 uPortal Developers Meeting Minutes

uPortal Developers' Meeting - IUPUI

September 19-20, 2005

Present At IUPUI

Via ISDN videoconference for all of both days at Rutgers

Monday September 19, 2005

uPortal 2 Status

Dan Mindler – Rutgers (via videoconference)

Slides

2.5.0 was available in May
2.5.1 RC1 planned in July but didn't actually happen

• Andrew says: just needs to be QA'd and cut the release

2.4.3 was available in August

• Rutgers is continuing to work on 2.4.x
• 2.4.4 for Novemberish
  • plans to have it running under Java 5
  • performance and stability issues Rutgers is continuing to work on
• 2.4.3 and 2.5.0 quickstarts are up

As of August 31, all known memory leaks found by Rutgers are plugged.

Dan explained the memory problems that they found (see slides) and how they investigated and fixed them.

finalizers:

• MultipartDataSource
• Xalan XRTreeFrag
• JavaMail 1.3.2 has finalizers – modified local copy

After a number of problems were fixed, there were still some things on the finalizer queue. Apparently the JVM was not letting the finalizer queue run often enough.

Too many open file exceptions

• • was it network sockets that did this?
  • no it was actual open files
• spelling engine used by UBC Webmail – Jazzy third party library was not explicitly closing files.

Rutgers then explicitly controlled when System.gc() and System.runFinalization() were called (the latter is only a 'hint' to the JVM, not guaranteed)

The tenured area of the heap would still start growing after a couple days.

The finalization runs were blocked and never finished.

Finalizer was stuck in a third party library that does SSH for their briefcase channel.

A lock release was not in the finally clause.

They recreated in their Quality Assurance the situation that was causing this, and they fixed the problem.

The channels use SoftReference caches; that is the worst case scenario for caching.

Latest issues they have addressed:

• introduce LDAP connection pooling
• Modify caching of layouts to share across users
• Tune caching of heavily used channels

Dan listed things they are going to try in the future.

Susan Bramhall – We don't use the (UBC) Email channel or the (Rutgers) Briefcase, we're on 2.5.0. We restart once a week on one server with a much lighter load.

User Experience Strategy

Jason Shao – Rutgers

Slides

He's been talking with B. Collier Jones about the following issues.

• Come up with a convention for publishing a channel parameter which indicates the "Channel CSS path, or media path, or image path"
• Enhancements they'd like to see – what's our strategy for advancing the user experience of uPortal as a whole?

• End users
  • The user experience should be flexible.
  • Should be simple, intuitive, discoverable, accessible; it should illustrate and support common/best practices, and provide a solid foundation for customization.
  • Common tasks should be simple and easy for a casual user to discover – common metaphors and defacto standards should be followed whenever possible
  • immediate, contextual feedback should be provided whenever possible.
  • user-oriented help

• Developers
  • Documentation should be clear, concise, complete, up-to-date, organized, and easy to find.

Collaborating should be easy

• has someone else done what you're trying to do
• easy to work with others to modify the framework
• easy to push back modifications back to the core.

Simple customization should be simple and not require major customization (skinning, branding, localizing, changing colors, text, language)

Extensive customization should straightforward

Accessibility, i18n, i10n should be strongly supported at all levels and simple to extend/modify

Supporting "marketing" materials to aid people evaluating/justifying uPortal should be available

Susan Bramhall – customization seems to be the part that needs the most attention.

Eric – As yet, up3 has not chosen what layout management implementation they're going to use, let alone what way they'll do preferences.

Jason wants to compile a library of use cases.

How can we think of some use cases we'd like to see going forward. What are the most important desired features that are actually feasible?

If your portal is mostly static, then Integrated Modes won't take long to render.

If your portal is highly dyamic, then Itegrated Modes is a lot of rendering overhead.

Jason – don't start with the implementation details, but rather describe the things the uPortal site community wants their users to be able to do.

Peter – but isn't that sort of defined by what the portal already does?

Jason – yes let's document and discuss what the portal does, and then go and discuss what other things users would want to do with it.

Eric – from a user management point of view, what are we looking to collect right now?

Do we want to render an Ajax API, and have drag and drop?

Mark is documenting an API for using URL's to do layout manipulation.

Andrew – this sounds more like a 'project' you're talking about rather than just a discussion.

Jason – once we have these use cases, we can then say: how do we make the features that the site is using, easier to implement in the first place? For example, what kind of minimum browser standards are people using? What about Ajax? What JavaScript minimums do you rely on?

Eric – there is a page buried on the wiki that asks for these use cases, and could ask these sorts of questions.

Jason – we need a feedback mechanism to inform our development.

Eric – Do we have someone who could spearhead a template for that (use cases) page and post emails requesting participation?

Jason – I will take some first steps on that, by the end of today.

Andrew – This can work well, but sometimes the process can fumble; the data gets collected in this way and then the information doesn't get used for anything.

Jason – in the next couple days we will be discussing technical details of layout implementation; this info could help.

Bill – Dan Mindler will continue work on 2.4.x releases. Jason will do the use cases thing.

Andrew is continuing to do the 2.5.x releases; he will plan on getting 2.5.1 out soon, will try to get it out before 2.4.4 appears.

Eric – let's do a BOF at Austin (December JA-SIG conference) about non-technical aspects of the user interface – that would be another place to advertise gathering requirements and experiences.

Andrew – well, don't say "non-technical" but rather "non-programming"; it's very technical and skilled labor needed to do this, just difrerent from Java programming.

Bill – now that CAS is in JA-SIG can we include it in quickstarts? How about rolling Rutgers' "alerts" system into the base code?

Eric – we need to have an agreement on the way in which bigger changes are made to the base code.

Bill – Are we at the point where these larger ideas are packaged as add-ons to the code, but we put them into the quickstart anyway to demo them?

Eric – The "alerts" thing requires a few small updates to the base code, so that's clumsy as a separate add-on.

Bill – Rutgers' alerts code delivers alerts to users in a variety of different ways. There is a subset of their code that has an alert manager, and a channel that allows creation and editing of alerts, and a mod to the login code that shows the user they have alerts. The individual site decides how they actually want it delivered (an email, another channel to look at them).

Peter – this is a good feature; many would like this in the code. But is there is a way to have this easily as a separate add-on? The changes to the base code are quite minimal but they are specific to the alerts feature; they don't support any other feature.

Eric – I remember this was packaged and put up on Confluence; why don't they do that again, and document the specific changes to the framework, and start a discussion on the list as to whether it should be integrated.

Andrew – we should be comfortable with having this kind of thing in the base code; that differentiates us from other portals because we are for universities and this is a higher-ed kind of feature (e.g. notifying students about an immunization program).

Bill – what about CAS – could we put it into the quickstart or base code?

Andrew – there is a CAS jar; questions go by sometimes. Why not just put the security provider and the jar (CAS client specific to uPortal) in the base. Is that what you are asking for Bill?

Bill – yes

Andrew – the first step – the CAS client jar, and three separate classes. The second step – there is a separate login screen in 3.0 (Peter – that is a temporary thing). Can we make it a permanent possibility and have CAS work with that if the site wants?

Peter – our general practice is that uPortal should choose the simplest alternative as the default. I don't want people to have to remove from the default installation, something more complicated, in order to get the simpler choice instead.

Susan – could we just put CAS in the quickstart as a demo, using the default layout. But the problem is then you have to explain to a beginner that it's not the only way and how to remove it.

Peter – I agree it's hard to explain (so does Eric.)

Bill – one good thing that CAS does is proxy authentication. So if CAS is in the quickstart, it could demonstrate that and single signon.

Peter – the login screen; it's a separate page like what CAS login would be. But a lot of people want the login on the portal screen itself instead. So we need to find out the reaction from the user community to such a change, before we do it.

Susan – why don't we just include the client for now and just discuss the other idea. Explaining something you have to proxy seems to be too overwhelming to a beginning user. (This was agreed to)

10 minute break

(The new Sungard/SCT) Permissions model

Keith Stacks – Sungard/SCT

Slides

Keith demonstrated how the Permissions Manager channel, as he has modified it, now works. This is based on the plans discussed at this meeting in March.

He also wrote his own GroupsManager to demonstrate the other features. Can use boolean expressions, with traditional group membership stuff as the leaves. He can get leaves from the traditional Groups Manager; or from PAGS (one of the LDAP derivatives) with relational operators on the latter. His permissions manager shows these resulting "filter groups" both graphically and as a textual expression.

He then gave a set of slides about hs proposed "Grant Restrictions" feature, which is for the purpose of flexible delegation of various aspects of uPortal administration such as channel publishing.

Limits the scope of the audience towards which an authorized user is allowed to perform some activity.

How? Add a new permission whose Target attribute is a fine-grained filter (group of users) where "filter group" is as described above.

A user's restriction set will be reflected in the filter hierarchy being used by an application

Multiple restriction sets are OR-ed together.

Applications will need an additional permission to preserve the restriction set that allowed the activity to occur.

The MANAGE permission grant's principal is the AND-ed set of restrictions.

Grant restrictions side effect: "cross-pollination"

when an unintended audience is the recipient of an object created by some activity.

If there are two or more different restrictions applying to the same person, and one of them is broader, then the user will be able to use the broader restriction in any relevant context, even if the original administrator who granted these permissions did not intend that.

The people most knowledgeable with the underlying concepts had an extensive discussion of what the terminology should be for these new restrictions (there was no disagreement about the usefulness or design of the feature).

Lunch break 12:15 to 13:15

one subject that came up during lunch chat is portlet support. Eric said he's mostly working on portlet support in 3.0 instead of 2.x. John Hare (U Wisconsin Madison) and Andy Gherna (U Illinois) said they were going to try to write some code as well.

Here are the preconference seminars so far planned for the December conference, JA-SIG is looking for other suggestions.

1 day on portlet development
1/2 day on Eclipse
1/2 day on CAS-3

end of break

Layout Management

Mark Boyd, SunGard SCT

Slides

Terms:

DLM Fragment Manager – Mark demonstrated how it works, which is much more advanced and flexible than the current ALM fragment manager. Fragments appear in the fragment manager list in order of their precedence. Those higher up have higher precedence. By moving a fragment lower in the list you are lowering its precedence with respect to other fragments. More information here: Defining fragments in DLM

As yet he does not have a link in the manager to go and edit the layout of the fragment.

He is working with Integrated Modes so that it will be able to be used for this purpose. DLM has the concept of preferences built into it. For example, column width is a preference value and can be set on DLM fragment columns. With Integrated Modes, there need to be two different sets of Preferences etc. (for the current user and for the one fragment they are currently editing) and swap between them.

Mark found a lot of parameters in the database related to the layout. They are processed in three places: UserPreferences, UserInstance, and in the ChannelManager. Mark would like to reorganize them to be processed in only one place.

Peter – when a new layout node is created, this information is temporarily shared between two entities, so that will be a little complication in making this change; but it is a good idea to do it.

Mark expects to have DLM and IntegratedModes working together in a couple of weeks, and the ability to use that to edit fragment layout shortly thereafter. (ALM-specific code; two more methods he has not implemented)

Andrew – is there any thought to doing a MUN-style mode, preferences always on?

Mark – That can be done in the stylesheets. MUN does this by the pre-existing parameter in the URL that tells whether the prefs are on or off, and forcing this always to be on.

Audience selection – Keith will be writing that and integrating it.

Permissions – doesn't have restrictions on it yet so that editing is properly delegated. (but someone else pointed out that the ordering of the fragments is important, so let's show them but not let them be edited.) Dan said there's a request to completely hide certain channels, e.g. Keith: can we just have a parm on a fragment to hide it.

Andrew – fragments are analogous to Sakai "sites", and that's what they decided to do.

Enhancements – currently there are two layouts in memory for each user. One is the one being shown the (Incorporated Layout Fragment), and another is the one which gets persisted to the database, (the
Personal layout Fragment) which has other things in it called "directives". Instead of having two whole layouts, just have references to other pieces that get followed only when needed.

At present there's no way to 'undo' a directive, such as "deleting" a channel from an individual user's view of a fragment. Maybe we could make this possible ("Reclamation channel")

Or, instead of using references, why not just load the PersonalLayoutFragment when they're editing and leave it out of memory at other times.

External layouts – layouts external to uPortal and pull them in when desired. (this was mostly for Sakai.)

Bill at Rutgers – we're using only the Simple layout, they like to use configuration scripts so they can push one button and suddenly implement everything for an install.
Mark – I visualize a way to serialize a fragment, and then a way to import that result.

Mark – this (DLM) is targeted for uPortal 2.6 or whatever release they're at when he gets this done.

Eric – some sites have all of their material as text files rather than only in databases, and use version control on it. Setting up a new server is a cinch. Right now simple layouts and DLM are the only relevant parts this cannot be done with.

Susan – how does this new fragment management model migrate from the old way?

Mark – I haven't written that yet.

DLM Fragment Manager I18n – right now it's fully internationalized in terms of development (but the text has not yet actually been translated).

They are changing the layous so that tab names will be localizable, as well as the channel names and channel titles. This needs to happen to the base channels as well. SunGard is not going to do this with XSL, but with the JSP model 2 type.

Andrew – why did you go this way instead of making some kind of portlet?

Mark – I want to go that way eventually. It is possible to do JSP without going model 2, and do everything straight out of the JSP. Instead, he wants to limit under-the-hood calls so he went model 2. Mark is very familiar with IChannel and all of the uPortal specific functionality and capabilities and hence created the model II JSP functionality as a channel type rather than as portlet for now. JSP is being used in place of XSLT for the internationalization features that are readily available via JSTL in JSPs.

Internationalization

Shoji Kajita – Nagoya University

Slides

Collaborative efforts required for i18n

• UTF-8 Transparency
  • Layout management mechanism must exist
  • There must be localizable resources via some kind of resource bundles and for the presentation layer, and in the database.

Shoji listed many aspects of the portal that need work. Due to his increased responsibilities he has not been able to spend much time working on the uPortal i18n in the last year.

i18n is highly dependent on the layout management model chosen

Much manual work is required.

Shoji explained how he has been doing the localization work.

• Check out of CVS
• Identify localizable strings and phrases
• Provide translations
• Build translation resource files
• Check back into CVS

Verify the user interface, and verify the translation quality on screen since there may be a case in which the same word is required to translate differently due to the context.

Also, we need a gatekeeper for each locale (or language) to keep the consistency of translation.

Web based translation site (in planning)

Non-technical translators can easily use it to get involved in translation works.

Translated resources automatically committed to CVS repository; this probably enables us to reduce such manual efforts to save our time.

Could be used for Sakai i18n effort also.

Should we make DLM the default for i18n efforts?

Andrew – it wasn't used for i18n in 2.5 because it seemed tough to migrate the existing content over. If the existing content could be migrated, then DLM could be used.

Peter – For uPortal 3.0, Eric is working on the LocaleManager; after that other aspects of i18n can be started in the 3.0 development. But I don't know when translation work can begin. So far we are only using the Simple layout manager. They are still discussing XLIFF to bundle up translated resources. But then, you have to code your own tool structure and to persist the information.

Susan – can it be kept separate from the XSL files so the latter can be edited independently instead of having to change N files (for N languages) to modify a single XSL script.

Peter – The reason for the XLIFF standard is to be able to add meta-information to the raw text of the translations. But it doesn't really help on the coding side.

Bill – three issues I hear

1. Internationalizing stylesheets
2. Leveraging standard Java support for i18n, such as JSTL.
3. What format should we store it in and how can we help the translators.

Mark said, don't forget 4) Parametric substitution ("There are too many ^0 entries in the ^1 table for user ^2"). XLIFF doesn't support that.

Mark – Jan Nielsen at SunGard SCT is doing most of their i18n work, and it seems to be along these lines, so let's take this discussion to the list, and get Jan and Shoji to exchange ideas.

short break

Michael asked about putting timeouts on http requests to channel content servers. Cornell had a problem with this and set a JVM parameter as an interim measure. Could there be a custom timeout per channel based on the rendering timeout? – Up through Java 1.4 the standard Http classes do not have this feature (Java 5 does have it). Rutgers was looking into using the Apache HttpClient lib to do such a thing for now. YaleInfo did something like this where they prefetch channels and were timing out the requests.

Groups and Permissions

Dan Ellentuck, Columbia U.
Slides

Separating Groups and Permissions (GAP) from the base code (status) and request comments on proposed changes.

Four subtasks

1. Creating a common services layer which Groups and Permissions can use.
2. Porting Groups API and Composite Groups service
3. Porting Permissions API and authorization service.
4. Porting of individual group service connectors Local, Person Attributes Group Service (PAGS), etc.

Four overall concerns during the conversion effort

1. A general review of exception handling policy
2. Creating some decent debug and trace level logging
3. Easier configuring
4. Issue tracking and documentation

The largest task was creating the common services layer. That's done.

By the end of this week (Sept. 23), Groups API and Composite Groups will be ported.

The third point will probably take another week (end of September).

Columbia uses File System group service; some people use PAGS; some use Just-in-Time LDAP (JitLDAP).

He estimated it would take about 3 to 5 days to port each one of those, and asked for volunteers to help.

up3 milestone 2 will be coming out very soon; see tomorrow's sessions

Questions
*Locking of groups; not so much the existence of a locking mechanism, but in the obscurity of how it is used. Sometimes a lock can be abandoned without being disposed of.
*The locking behavior is the responsibility of the group service itself. Others calling upon the group service should be able to ask it to fix some of these issues. (Keith – it should be possible to ask whether a group is locked, without trying to get a lock)
*Composite design of the group service. This anticipated assembling a group service out of components which themselves could be assemblies, e.g. requests from a remote group service. Now recursive composites seem overkill. Code would be simpler and easier to maintain if it didn't have this.

Andrew – but remote group services seem like the wave of the future. It's part of the Sakai model, I think.

How about the Grouper Project. Let's make a connector to Grouper from our groups model.

Bill from Rutgers re Maven – ant seems to be good enough, but there are things about Maven that are nice.

Dan – should we wait for Keith's modifications before porting PAGS onto this?
Eric – waiting until a later point to add the various composite services, will not hold up Milestone 2 of uportal 3.
Keith – it would probably make good sense for me to be the one to port PAGS eventually, but I don't know when I will get to it.

Eric – what's the status of the API compatibility with it? Once this gets used and tested, could we get it moved to 2.x as well?

Dan – the new code is configured with Spring, so can we require up2 sites to shift to that, or do we have to support the old properties files?

Eric – PersonDirectory was a similar situation – for config in up2, we removed the original code and rewrote an adapter to read the old configs and convert them.

The rationalization for both of these 'detaching' projects is to move GAP and PersonDirectory up to the container shared library dir, so that other contexts, for example portlets, can use them as well without having to have separate copies.

Dan – what about using Hibernate, could we require that?

Eric – for up3 we're moving to Hibernate for the persistence layer. for GAP, I don't mind what you use.

Peter – so the API is going to stay the same?

PersonDirectory

Eric Dalquist, Unicon
Slides

Ernest at Unicon did the work of moving it into a sandbox environment. It needs a little more work on its Maven build. Then it will be included in uPortal 3. Possibly also in 2.6.

As time goes on, version management and library management will become a little bit more of an issue for uPortal.

Maven

Andrew Petro, Yale
(Andrew did not have slides)

(Andrew recommends adopting Maven for uPortal builds)

first there was shell
then there was make
then there was Ant – it is procedural – telling it HOW to do things.
then there was Maven – it aspires to be philosophically different

Ant is about defining targets to do stuff

Maven – project comprehension
POM = Project Object Model

You put all this metadata in that tells what is true about your project.
Then you tell it how to attain goals. (goals are shared)
Maven contains a massive library of predefined goals, you don't have to write it in each project.

https://clearinghouse.ja-sig.org/wiki/display/CAS/Home
(sorry I forgot why that link is there – Mike Oltz)

It also manages dependencies
This is the Maven repository of all jars of all Maven projects.
Maven wants to handle multiprojects. So POMs can be nested, as it were. You can have dependencies on other jars.

Susan – one of the biggest problems in deploying uPortal is the xalan/xerces jars, where they are, what JVM you are running them under, etc. Does Maven make that easier? Andrew – no.

Peter – what is different for the developer when using Maven? I don't know if there are predefined goals to do some of the things I want.

Eric – has used Maven in the past. Maybe there's some more overhead but it does the same changed-file checking that Ant does.

Andrew – there's a Maven plug-in for Eclipse that will make it look just like the Ant one.

How do you handle multiple source directories from Eclipse? Well a single project can have more than one source directory; or you can have one project per source dir.

Bill at Rutgers – Andrew are you suggesting Maven for everything? What if the aspect of it we really need is just dependency resolution? There are other packages that do dependency resolution the same way Maven
does.

Andrew – I think Maven is generally useful.

Bill – what are the drawbacks going to it?

Andrew – there's a learning curve.

Eric – there's the one-time cost of getting the project xml set up

Peter – moving things around is the biggest cost, if you want to use multiple projects.

Eric – if we don't split up the source tree then the only advantage is the dependency resolution.

Susan – it would seem more logical to wait until uportal3 to adopt Maven.

Andrew – it would have some advantages for uportal2. Let's discuss it for 2.6.

Susan – that (rearranging the source files) would make migrating from 2.5 to 2.6 very hard. (for schools that have modified the base code; as many larger sites have done)

Eric – probably good for uportal3, but using it for uportal2 is likely to upset a lot of schools. Andrew, do you want to start discussion on
the list about the latter? Andrew: Yes.

end of day

Tuesday, September 20

h4,uPortal 3 demo
Eric Dalquist, Unicon

(live demo rather than from slides)

This demo has most of what will be in Milestone 2, coming out in the near future (planning for end of next week, i.e., Sept. 30)

(Peter: One of the goals is to have compatibility with existing stylesheets; that's why the layout looks familiar.)

The Portlet Manager is essentially finished (replacement for Channel Manager).

Can add multiple instances of the same portlet. Can flexibly map between one portlet mode and another, so you can substitute the modes. Can flexibly map Application User Attributes to uPortal User Attributes. These make it easier to use third-party portlets in uPortal.

Has a 'preferences' editor for what channels called 'parameters'. Can
reset Portal prefs, which can have multiple values and can be read only or read write. The three sources of prefs are – the portlet xml, the publish-time specifications, and run time values that can perhaps be changed by the user. Note: it was pointed out that if a preference is read-only in the portlet xml, you should not be able to override that at publish time; the developers had not thought of that so they will change it.

This Milestone also has the ability to manage the layout a little bit * can move columns, channels, and tabs around, can't yet add content through UI. The demo showed simple layout with integrated modes.

As yet, uPortal 3 runs only under Java 1.4 and you must remove those XML jars from the endorsed directory of your JVM.

An Alternate Layout for uPortal 3

B. Collier Jones, Virginia Tech.

Static mockups (i.e. just graphics) of suggested minor changes of the flow for uPortal3 UI.

Slides

In upper right is Settings, Help, Logout. Settings has a dropdown list of many alternatives, reset, edit layout, so on. Clicking any would take you to another screen that has all the choices on the left, and the specific setting you want to make, editable on the right.

In Collier's design, there would be more things that you could choose from the title bar of each portlet, than there are in any of the present designs.

He wants there to be 'subtabs'; choose a tab and down the left side is a list of the subtabs of that tab; for clearest design there should only be a few subtabs when they are used.

Once you have chosen content to add, it will show targets on the current tab, and you choose the place to insert the new portlet the other content is shown but disabled; you can switch to a different tab while in this mode.

He also has a concept of "pages" whose content is something like a tab, but you get to it by way of a link inside a channel rather than
clicking on the tabs.

We should only implement features that will work both with and without JavaScript.

The feature of his design that he most wants to see implemented is the Settings portlet.

Eric – that would be conceptually cleaner for users and site admins.

Andrew – but it would require a new way of configuring the Settings page itself; how could we use the current GUI design to configure the Settings page the same way we configure tabs?

uPortal 3 Navigation

Peter Kharchenko, Unicon

Slides

Discusses some parts of the internal code in uP3 that manages URL generation and interpretation, and how those URL's should look and function.

AJAX (Asynchronous JavaScript with XML). Not an API of its own, but a variant way of using existing web development technologies to create a smoother user experience.

If the uPortal project uses Ajax, it will be as gradual improvements with Ajax rather than sweeping adoption

• Among other things, it makes possible single-portlet refresh without refreshing whole screen
• Lazy loading
  • can do theme transform on the client
  • may look weird
• Drag-and-drop portlet management

Jason: Be careful not to go overboard on server-side profiles (which are part of the Ajax paradigm); we could wind up with many different versions of themes.

Mark: Could we put in some 'test the capabilities of the browser' JavaScript during login, look at what features are implemented, and use that to determine what can be used in the responses?

Peter: One thing we use the profiles for is, people can choose different profiles based on what kind of browser they log in from (although most sites don't support this). So it's not just for technical reasons. I think we should implement 'smart' stylesheets, but yes we should try to keep their complexity from becoming overwhelming.

Setting aside the Ajax suggestion, let's now talk about *

URL construction and processing:

• Parameter processors
  • independent, serving specific framework components
  • Portlet, UserLayout, Transformer, persistence controller

• URL constructors
  • Independent, but can make use of common base syntax provider to have a common style
  • can be aware of internal states of other components

• URL decorators
  • automatically invoked constructors are used to reflect states of framework components

*Configuration-specific URL syntax

• • can be very flexible (human-writable URLs)
  • may not implement all the features for different components
  • can be configured on per-component basis
  • not packaged – a set of parameter processors and URL constructors that need to be injected/configured within the portal context

*uP2 syntax support (our initial target, so we can support up2 stylesheets)

• • can and will be implemented
  • limiting in a number of cases
    • example: explicit structure and theme separation

*default URL syntax

• • purpose: bookmarking. URL chopping
  • example: http://site/portal/myTab/mySection/
  • Implementation
    • decorator that's aware of the structure transform preferences and their meaning
    • parameter processor that analyzes the secondary path

Generalization: stylesheet description declares navigation params/attributes (hard to do with as nice of a syntax as above)

*Encoding naviation state: problems

• • What is "myTab"? Is that a tab name (English only) or a tab id?
  • How to separate request path element processing
  • portal context is currently resolved by the first path element
  • is there a nice syntax for other navigation state variables?

Susan: What is the implication for Sakai?

Andrew: Originally Sakai was going to be some neat channels running under uPortal. But they needed this more flexible navigation before we could have it ready, so they went and built their own container.

(back to Peter)

• Returning to a bookmark: authentication
  • Personal tab -
    • need to know what user the tab belongs to http://portal-user/myTab/mySection
    • need to authenticate before content can be shown
  • Guest-accessible portlet (bookmark by fname)
    • can be shown initially without additional authentication
    • User can authenticate later

What if one user sends a link to a tab to some other user, and the other user logs in. Well that will work only if both users have the same tab; e.g. if it's a fragment they both have, they will get their respective individual view of that tab.

There was an extensive discussion of generic and user-specific
navigation URLs. I did not take notes for most of it.

Andrew: If a tab is created by oneself, then a link created to point to it should have the userid in it (as had been shortly before discussed by Peter and others); if a tab is created by the site, then the link should not have the userid in it. Would a mapping from a userid to a huge number be sufficient to meet privacy law requirements?

The outcome of the discussion was that Peter would forego the human-readable URL's for now and just use internal ID's.

break for lunch

during lunch break, B. Collier Jones asked that fnames should have a specified character set rather than allow all characters, so that they can be used in selectors for CSS: letters, numbers, hypens, and underscores. He is using this capability to have stylesheets for specific channels.

Mark: fnames should be unique. Eric: in version 3 it is required to be; the db column is marked as unique. As yet, no restriction on the character set.

Andrew: The way it works at present, if an fname is not unique, Ken (Weiner, former project lead for uPortal) once said it is supposed to resolve to the most recently published channel with that fname.

end of lunch break

IChannel support on uPortal 3

Peter Kharchenko, Unicon
Adam Rybicki, Unicon

(This session had been listed in the agenda as presented by Peter, but Adam and Peter agreed to have Adam do most of the presentation.)

Slides

Adam: Many sites have made major investments in developing IChannels, so they are especially keen to have a smooth migration path to version

• Unicon has been porting to portlets for a year or so.

Downward compatibility

• When uportal 2 was being designed, there was only one uportal1 site in production (University of British Columbia). When it was released there were fewer than five.
• We now have hundreds of sites in production.

How do other kinds of projects support downward compatibility?

Tomcat will not disable support for "old" servlet and JSP versions.

EJB3 greatly improves ease of development, but no container vendor will abandon support for 2.1 or older.

Commercial vendors will not abandon their proprietary (pre-JSR-168) interfaces. (Vignette, WebSphere, Oracle portal). secondary reason: Portlet standard is not as yet comprehensive or flexible, you can do more with proprietary interfaces.

IChannel is an example of a mature interface which in some respects may be superior to JSR-168.

What level of compatibility?

To be 100% compatible with 2.x, 3 would have to include the entire 2.x framework.

There are many channels out there. How many rely on internal framework classes?

Probably the least portable channels were developed by developers most familiar with the framework internals.

Proposal: ask the community to send us all of their "import org.jasig.portal.*" lines of code. (this is a serious request, not tongue-in-cheek). In other words which framework classes are being called from channels.

How will common framework services be accessed by uportal 3 portlets? We already know that PersonDirectory and Groups will be easily accessible.

Alternatives

• ChannelAdapterPortlet
  • provides channel lifestyle, serialization

• Standalone uPortal2 instance
  • tweaked to deliver standalone channels
  • internal request dispatching
  • communicates with the uPortal3 instance via WSRP

• Native support for channels in Uportal3
  • But ChannelAdapterPortlet would be cleaner

Other ideas?

Adam asked what other open source portals are doing with respect to this. Nobody knew.

What about privileged channels (which have greater access to framework channels)? Some worried this would be hard to support, others thought not.

ChannelAdapterPortlet (the most probable approach)

• Maintains channel instances
  • Similar to ChannelManager in uP2
• Takes care of
  • Channel lifecycle
  • Parameter processing
  • Serialization
  • Caching

Provides access to framework (capabilities?)

Instance management

• IChannel: one instance per portlet window
  • can be maintained in portlet session
• IMultithreadedChannel: one instance
  • will rely on channel to clean up internal maps based on the events sent by adapter

Channel lifecycle

• init() = setStaticData()
• processAction() = pre-setRuntimeData()
• render() = setRuntimeData()+renderXML()
• portal events

Parameters

• processing is straightforward
• no more baseActionUrl
• make all URLs actionURLs?

Access to uP2 interfaces, support classes and services:

uP2-api.jar + uP3 implementations

• Basics
  • Channel interfaces
  • ChannelRuntimeData, ChannelStaticData
  • IPerson
• Further
  • IAuthorizationPrincipal
  • ICC Registry, JNDI context (wait until we have some functionality of IPC implemented)
• Static services
  • GroupServices, AuthorizationService, EntityNameFinderService, (PersonDirectory)
  • RDBMServices
  • XSLT utils
• CAR support
  • CarServices
  • Packaging/deployment

Peter: It will be necessary to recompile all custom channels to make them work in uPortal 3.

Mark: In JSR-168, there is no mechanism for extending portlet.xml is there? (answer: No) so there is no direct analog to a CAR descriptor.

Mark: Ideally we should urge people to move to JSR-168.

Peter: I am wary of the idea discussed earlier of running two containers as you have to duplicate so much code and functionality.

Andrew: If we have to recompile anyway, couldn't we also require minor adjustment to the source, and then we could support 90% of the functionality the old way.

uP2 to uP3 Migration

Eric Dalquist, Unicon
(no slides)
(see session on Layout Management for definitions of "LM" abbreviations)

• Group configuration
  • Rolled out PAGs service
• Layouts
  • Need to determine uP3 layout management
  • Theme XSL updates?
• Channels
  • Document 'public' uP2 APIs
  • Static services facades
  • IChannel adapter

Most important consideration is copying the layouts in the database; user information is probably not as important. Andrew: skin, channel and portlet configs are important.

Eric: Group membership info is important also. And configuration files.

Michael: We found in going from 2.1 to 2.4 that end users won't care whether it's difficult to carry their custom layouts forward; they want it anyway.

Eric: There is an WSRP consumer and producer already in uP3.

Adam: There was a simple port of CWebProxy to uP3 that was proven to work; and there are two other implementations that would also work.

Peter: We have support for the Simple LayoutManager, but not yet ALM and DLM. It might be straightforward, but there's definitely some effort involved. It may not be sensible to port both of them. But the degree to which each of them is finished and polished in 3.0 depends on demand.

Susan: If 3.0 GA comes out only with SimpleLayoutManager, it will feel like a backward step, so people will be reluctant to move to it.

Andrew: It is nevertheless valuable to have a release without ALM or DLM, so there will be a code base to then put one or both into.

Peter: How do we determine priorities for upgrade tools? Which should be written first?

Andrew: Given that this is a completely new release, I don't think it's
necessary to write tools for configuration files; a deployer would do it incrementally by hand anyway. What we really need is documentation, separate from the config files themselves, that tells how to do it by hand.

Eric: We really need to decide what path we will take for Layout Management in uP3.

Peter: We talked about a feature list for what Andrew calls the "CLM" (ComplexLayoutManager, a tongue-in-cheek name for a future LM).

Eric: We have SLM working in uP3, we may do a Release Candidate (RC) with that only, then go on to make another layout manager.

Peter: Perhaps porting both ALM and DLM may not be conceptually difficult, but it may be a lot of work. It is likely to take less time than writing a new LM from scratch.

Mark: The Immutable and Unremovable attributes are the primary differences between the ALM and DLM database structures. We could probably port either ALM or DLM data in uP2 into, say, DLM in uP3.

Peter: I'm not sure how ALM restrictions could port to DLM. Perhaps we could make some simplifying assumptions.

Eric: We would like to have an RC or General Availability (GA) by December.

Mark: Do up3 layouts still have "ChannelElement" class?

Peter: Yes. Most importantly, PortletWindowID element class is a new thing in up3. This includes lots of things that used to be elsewhere.

Peter: There is a "LegacyLayoutSource" that allows you to wrap the old ones. But if we have a completely different persistence store, migrating data will be tougher.

Mark: We ought to add the internationalization of folder names, etc. at the same time as the migration.

Decision: We will roll for December with SLM only.

short break

Enterprise Integration

Adam Rybicki, Unicon
Slides

Sakai has learned a lot from uPortal and have improved on it in certain ways. They have defined integration points for enterprise software (that is to say, places within the Sakai framework where it makes sense to be able to access and use enterprise data). uPorrtal should consider such an effort for itself.

Proprietary EAI solutions were popular in mid-1990's.
Value: pre-built connectors to most common enterprise systems
Custom connectors expensive to build.
Locked to a specific programming language (often C or C++).
Security, if any, seemed like an afterthought.

Enterprise Integration Challenges

• Communication
• Connectivity
• Transformation
• Service-Oriented Architecture
• Portability
• Security
• Standards

Messaging: JMS
Web svcs: XML, SOAP, WSDL, UDDI
Transformation: XSLT, XQuery

SOA:
Portability: Java, HTTP, XML, SOAP on Windows, Unix, mainframes
Securty: SSL, certificates, signatures, SAML plus all of the WS-* security standards

(He showed an SOA diagram demonstrating conceptually how it works)

Enterprise Service Bus

• Not necessarily a message bus
• Evolved from various service protocols
• ESB provides the backbone for building an enterprise SOA ...

New Java-Specific standards

• EJB 3.0 – ease development over 2.0
• JBI (JSR 208) 1.0 – Java Business Integration interface (answer to the proprietary nature of EAI interfaces)
• BPELJ – a Java spin on the business process execution language (XML for workflow)
• JDBC 4.0 – ease of development improvements

EJB 3.0

• simplifies programming
• eliminates requirements for Home, Remote interfaces and deployment descriptor
• Allow transaction demarcation with annotations
• Remove requirement to implement boilerplate methods ejbCreate() ejbActuate() ejbRemove()
• Continue support for 2.1 and older

JBI

• Standardizes EAI APIs
• Prevents vendor lock-in
• Helps with building SOAs
• Too new to know its potential for success

JDBC 4

• can use Generics and Annotations
• SQL 2003-compliant XML data type
• Connection.isValid()
• Connection and Statement pooling is now a part of the standard
• Specific SQL exceptions: transient and non-transient (transient might succeed if retried)

Java Tools

Java 5 is needed for all standards that require annotations. Annotations are a special kind of Java interface that is used to put meta-information on packages, classes, methods, etc. The information can be used by the compiler, an IDE, a javadoc builder, or at runtime.

Annotations make EJBs much simpler to write

Java Web Services Developer Pack 1.6
(this is more oriented toward SunOne than toward generic containers)

University of Colorado has designed and are implementing a pragmatic Service oriented architecture

• based on WSDL standard
• Using Glue from webMethods (now replaced with the next generation
  product called Fabric)
• Glue simplifies WS programming greatly
  • no SOAP, WSDL, or XML
  • POJO (plain old Java object) to WSDL and WSDL to POJO conversions
  • includes a SOAP server
  • demo?

University of Illinois developed OpenEAI

Conclusions and Recommendations

• Most importantly, access enterprise directly rather than through a shadowed copy. Not by raw API's but through an ESB.
• Choose standards-based, independent solutions.

(for other conclusions/recommendations, see slides)

What is Sakai doing?

• Has an Integration Discussion Group dedicated to Integration
• Developing a list of use cases

Current Sakai integration priorities:

• Provision of Sakai information by populating a database
• Provision of Sakai information from Live external information systems
• Integration with a portal
• Stable, Documented Framework Architecture
• Data recovery operations

What Should We (uPortal) Do?

• Pick an integration technology to use (WS, JBI, EJB3, OpenEAI)
• Define integration data points. Examples
  • Persons
  • Groups
• Define a standard method for portlets to integrate their data. examples
  • calendar events
  • announcements
  • notifications
• Define the objects in XML
• All the integration points should be bi-directional
• Authentication!

Andrew: there are other relevant things Sakai is doing. There is that formal discussion group. But there is also a developer named Steve Givens, who, in an informal exploratory manner, is using Axis to produce Java Web Services descriptors, so just about everything in Sakai can be done using WS.

Adam: One difference is that vs. Sakai, uPortal is still only a framework, rather than a collection of actual tools. So we have fewer integration points in the framework itself. It is the applications (portlets) that have more need of enterprise information.

(The group agreed that at least, the uPortal development group should recommend a particular integration technology for portlet developers to use.)

Andrew: Web Services makes sense in the framework particularly to the extent that we are a portal. It makes sense to have a WS endpoint to publish channels, or create users, or push fragments.

(The consensus appeared to be that WS should be recommended as the integration technology of uPortal.)

uP3 Sakai Integration Update

Andrew Petro, Yale U.

(no slides)

Andrew attended a meeting on this topic two weeks prior.

Sakai does not use uPortal inside of it (some people think or have thought it does).

One thing we did at that meeting is to get uPortal 3 and Sakai to work in the same servlet container.

One thought is to use uPortals facilities to provide the users and user attributes for Sakai.

Short-term, we put a layout with a Sakai tab on it, but the tab is empty. We put a layer in the rendering pipeline that gets Sakai output and inserts it as the content of the tab.

Susan: Why isn't that a channel, why does it have to be in the pipeline?

Andrew: Because you can use more than one IFrame, and it fits better into the uPortal layout, has a uPortal look and feel.

GAP, and portlet subscribe system that Eric showed earlier for up3: It needs to be able to plug in multiple-source portlets, not necessarily the current portal instance; may be via WS. That would be the next increment after the rendering layer.

Andrew: How do we authenticate WSRP? There will be coordination between Michael Ivanov and the Sakai WSRP group to work that out.

Shoji serendipitously had a diagram from a talk Charles Severance gave at Nagoya U, that Andrew used to explain the places where uPortal 3 and Sakai would interconnect in the second iteration of the integration.

The broader model described is an ongoing project that won't be finished by December.

uPortal 3 Testing

Eric Dalquist, Unicon

(no slides)

Following the same definitions of release levels as in uPortal 2.

What we really want right now for Milestone 2, is to get one or more test instances set up on the Clearinghouse machine, for people who aren't motivated enough to run the quickstart, to go and use it.

Please try this milestone one way or another.