September 2006 uPortal Developers Meeting Minutes
uPortal Developers' Meeting
September 12-13, 2006
University of Wisconsin at Madison
Attendees:
Eric Dalquist UW-Madison
Jim Helwig UW-Madison
John Hare UW-Madison
Ozzyie Chen UW-Madison (came in during the project management discussion)
Cris J. Holdorph Unicon
John A. Lewis Unicon
@Andrew Petro Unicon
Gary Thompson Unicon
Peter Kharchenko Unicon
Jonathan Markow JA-SIG
Michael Oltz Cornell U (these notes taken by)
B. Collier Jones UMBC
Jan Nielsen SunGard Higher Education
Jim Farmer IM&M (Monday only)
and via videoconferencing:
William G. Thompson, Jr. Rutgers (left at lunchtime Tuesday)
Dan Mindler Rutgers (was not present Monday morning)
Jason Shao Rutgers
Dan Ellentuck Columbia (at Rutgers)
Susan Bramhall Yale (at Rutgers; arrived after Jonathan's formal remarks, left at lunchtime Tuesday)
Roger Despres Yale (at Rutgers; arrived after Jonathan's formal remarks, left at lunchtime Tuesday)
Nasser Salomon UCal Riverside (at UCal Riverside, with two other people)
Guide to some uPortal-specific abbreviations:
SLM - simple layout manager, software behind the earliest type of uPortal layouts.
ALM - aggregated layout manager, introduced in 2.2. allows for 'fragments', centrally managed portions of a user's layout.
DLM - distributed layout manager, introduced in 2.5. similar features to ALM but more sophisticated.
GAP - Groups And Permissions, a hierarchical way of managing user authorization. created for uPortal by Columbia U.
Other recurring related terms:
Academus - a suite of software for higher education which embeds uPortal, made by Unicon.
ACEGI - a specification and software for doing authentication and authorization, embedding CAS
Ajax - a way of combining existing web development technologies to do more UI processing on the client and make a more responsive experience for the user
CAS - Central Authentication Service, single sign-on software developed at Yale but now under the aegis of JA-SIG
JSR-168 - specification for 'portlets', morsels of functionality to be imbedded in any portal supporting the standard
JSR-268 - revised version of the above
Luminis - a suite of software for higher education which embeds uPortal, made by SunGard Higher Education.
Sakai - a collaborative learning environment being developed by a consortium
Spring - a way of flexibly weaving together the pieces of an application at runtime startup from specifications in XML files, rather than by editing Java code or using Java properties
WSRP - Web Services for Remote Portlets; a way to interact with a portlet on another host by way of Web Services.
These minutes include various company and product names which are the trademarks or registered trademarks of their respective corporations.
MONDAY
Introduction - Eric Dalquist
Message from the JA-SIG board - Jonathan Markow
On the one hand, we've been a tremendous success in deployment, over six years of conferences and development meetings. There are some university developers who have been developing really good processes for keeping 2.x going. We have lots of happy users.
On the other hand, the higher education community is concerned about the long development process for 3.0. There is increasing perplexity from them about the future of uPortal. The perception is that things are not moving along.
One big cause is poor communication
The community is looking for stronger leadership from the JA-SIG board
The rest of the talk will be some of the things they have been thinking about
Why did this happen?
a) Sakai's primary concern was of necessity their collaboration and content management software, and they were not able to concentrate as much on their relationship with uPortal. It did not help that there happened to be no core uPortal implementers among the Sakai schools.
b) There was not ideal coordination between the university developers and Unicon.
c) Board oversight and accountability was not well defined
d) Process of accepting external grants and impact on developer community was not thought through
e) Differing viewpoints on what constitutes a deliverable 3.0 release.
Things the board is considering:
(A) At Vancouver Jonathan said there should be a minimal release of 3.0 without upgrade path and without advanced features. He now says this needs to happen before December conference.
We need to make uPortal competitive as a next-generation portal. Improved presentation, flexibility of presentation, Ajax etc.
We need to communicate exactly what 3.0.0 does and does not do, and what will be appearing in 3.x thereafter.
(B) Board will look at processes for project governance. Include strategic stakeholders in direction setting process to decide what's important. Define board involvement. Include developer input.
(C) Change the development process. We want to ensure that there is increased communication about what is going on.
(D) A software project manager for uPortal. Full oversight of the technical roadmap. And have release managers for each release.
(E) Expand the developer pool. Perhaps we could raise funds for a community developer fund, bring in developers from third-world countries, or fund transportation to developers' meetings.
(F) Improved JA-SIG organizational development. Have institutional membership?; this is a sensitive issue. Up to now JA-SIG has been very inclusive. There has been a lack of hierarchy in our gatherings and people find this very refreshing. How do we not compromise this? We don't want a membership fee to exclude participation by non-members. We don't want to have individual memberships or discourage anyone. Expand vendor sponsorship.
(G) Could we get a full-time executive director position for JA-SIG? Want to improve communication with cousin projects. It is not possible to keep up with the level of activity that is now there, and at the same time have other full-time jobs. The JA-SIG directors did not meet that often until recently; now they meet monthly by videoconference.
(H) Have a more active and more informative web site.
(I) Have the board more representative of the community. Perhaps elected by them. (Up to now it has been informal. When a board member can no longer participate, someone is sought who is active in the uPortal community, has a vested interest in its future, and is aware of the issues and what is going on, and they are invited to join the board.)
Summarize:
We need to release 3.0 in some form. And make the incremental improvements by a clear process rather than informally as in the past. Broaden community support and input. Improve the JA-SIG organization and leadership.
Jan: Has heard some of these concerns from the user community. The community is sensitive to those issues and there is some self-reinforcement. What this is, is growing pains. The more recent uPortal adopters are smaller schools with smaller developer groups and less in-house expertise than early adopters. If we release 3.0 without advanced features, etc. then some will not want to go forward to that. I think it's good to bring something out to demonstrate progress but we must not lose sight of the upgrade path.
Jim F.: We use 2.2 at IM&M. For what they needed other products were not as good. The two things that 2.x has had was great project management under Ken, and the stability testing and improvements that Rutgers and others did. In early 2003 it was very clear what was going to be done for 3.0, but as time went on it became less clear.
Most of the newer users other than Luminis are community colleges and small colleges.
uPortal has an incredible history of success. Sakai project had more processes in place.
Jim F: GAP is unique both to the uPortal and to higher education.
Moodle? They did the best work in considering pedagogy in their design. They asked faculty members what they needed most - a portlet that tells students where they are in the learning process.
Collier: What will we call this? uP3.0.0 has fewer features than 2.5, how will that be received?
Jonathan: We need to let the community know exactly why we're doing this. (to demonstrate that 3.0 is not moribund and progress has been made)
Jim F. wants 3.0 because (a) IM&M has the simplest uPortal web site, so they can try it without needing the advanced features. (b) There used to be one or two presentations per month outside of the user community of what uPortal is, that's not happening any more. The early 3.0 could be used to resume that.
Jim F: In open source you can't identify when some things are going to be done. Moodle has a schedule through 2008 but they have funding. We need a list of the order in which features will be implemented. We will eventually need some intense work on stability.
The very first release (by December) should be called a Release Candidate.
Eric: We could use some time here at the meeting to discuss some of these issues, especially what is a 3.0 release?
Andrew: Frequent releases are a good thing. It would be great to have something by November. It is critical that we bear in mind, "what makes it uPortal?" What is 3.0 so far, it's a JSR-168 container and a few other things. We should talk about an "upgrade path" rather than "backward compatibility". We do have the neat complicated GAP already in 3.0. uPortal is the portal by for and about higher education, we need that to be present. The real uPortal would have an upgrade path, the new requested features, and the old requested features. These need to be a large part of our presenting the prerelease.
Jonathan: A bare-bones 3.0 is not the only thing that's needed now. The community wants to be given a clear view of the path ahead.
Jason: a) One concern when communicating this message, what is the future of the 2.x branch? What is the transition plan? This is crucial for people planning to adopt now. b) We need to get community participation in testing and shakeout of 3.x.
Bill: a) agrees with having a clear message for potential adoptees 'which do we adopt now?' b) When 2.0 came out, there were schools lined up to adopt it. Right now there's nobody lined up. c) Migration is a key aspect, and building the comfort zone.
Susan: uPortal 3.0s attraction will not be the internal refactoring, but the new user interface on it. That will push people to want to get to it. If that's not there, people will be less interested. (Bill agreed)
Eric: I would like to know three things from the board: a) A definite time when the 3.0 release needs to be out? That sounds definite.
Jonathan: I agree that what will motivate 2.x users to migrate to 3.x will be that richer feature set that we intend to bring in early for that very purpose. 2.x schools are not interested in the refactoring, but in the new features. Unicon is working very hard to put that out there. But we have 2.x developers continuing to work on that. We need university developers to help with the 3.0 work, but they might not be on board until there are more features.
Eric: b) We need to know what we are going to call the November and January things? c) Does Eric have free reign to go through and prioritize for that? How will those decisions be made?
Project Management - Eric Dalquist, U Wisc Madison
"Software Project Manager" - We had an interim arrangement - just having release managers per dot release. (2.4, 2.5, 2.6, 3.0). Eric feels the community is less connected because of it.
Part of the discussion is to have some group of interested parties including release managers, board representative, and this project manager. Project Manager facilitates communication, and has a broad high-level view of what is going on.
Jason: I have concerns with coordinating community involvement. As this is open source, the project manager will have no authority to give directions or prioritize work. More control over submissions and less control over work priorities.
Eric: What we will be looking at for 3.0 is time-based releases (rather than feature-based "whenever x feature is done"). There needs to be someone who points out these things.
Jason: Ken was paid half-time to do this stuff. Will this be funded?
Jonathan: Yes, I don't think it's practical to expect it to be volunteer.
Susan: Is Eric the 3.0 project lead at present?
Eric: Yes that was decided in Los Angeles (developers' meeting at Unicon)
Jonathan: Get more community involvement in there. There needs to be a mechanism, a process, for that decision making. Perhaps there also needs to be a "product manager" more involved with community concerns rather than coordinating the developers.
Jason: Ken grew into his role. I think uPortal community is uncomfortably large for one person to have this level of oversight. Perhaps a committee would be better.
Susan: The most compelling reason to have a single person at the top, to make sure that the different threads are cooperating, so that improvements and bug fixes happen everywhere they are supposed to happen.
Jason: Some of our branches are created too early, so we have to maintain too many branches at once.
Andrew: When we did 2.5, they should have tagged and released the HEAD and declared it as 2.5, but not branch.
Eric: I've heard the question, are we too big to have one person doing it? Ken was not familiar with every bit of code. We have de facto component owners, such as Dan Ellentuck (GAP) and Mark Boyd (DLM) who take some of the burden off the one person.
Jonathan: I don't think we'll ever get so large as not to be able to use a specific person as a project manager; that person's role will change, but we need to have someone who has a broad view and monitors progress, promotes communication between developers of various pieces, and ...
Jason: Are we setting ourselves up with a single point of failure; what if the person leaves?
Jonathan: If there is a defined role, then we make sure they are not a single point of failure. There need to be other people who have some idea what that person does and can step in.
Andrew: Having a person working this role, in the near term that's unambiguously a good thing. So do we define what the role is, then we go and find the resources to make this position happen.
Jason: Should we first make a list of our pain points, and then decide what we need to do about them.
Jonathan: Are there more points that could have been addressed in my talk?
Jason: Shouldn't we also ask the community what we can do to fix our problems?
Eric: I like the idea of listing what we're missing.
- board <-> developer information sharing.
- have a group to inform the development process
- developer <-> project manager coordination
- release manager is there to package things up
Jonathan: I think we need the commitment from the release managers. If we don't find a release manager, then we escalate it to the development steering committee and they have to solve that. The person who gets asked to do that, must have the technical skills to do it.
Andrew: We've talked before about coordinating work on things other than the core, e.g. the Clearinghouse contents, and various channels, (e.g. some people want a better webproxy portlet) things get contributed, but then there are not "evangelists" to develop interest in using it.
Jason: Perhaps what we need is a working group first, that does the stakeholder sort of things and governance, and then the project manager comes along later.
Eric: What would the board like to see in a steering committee, its composition? Let's try to put together something in the next couple of weeks. We would want to have the release managers on it at least, from the developer side.
Jonathan: I don't know how quickly it could be done to put together.
Jim H: Are we going to look for the board to drive the formation of this steering committee?
Jonathan: Yes.
Jim F: At Georgetown they call such people "coordinators" This softer title tends to work better.
uPortal Development process - Jason Shao, Rutgers
Some suggestions were made about a more defined uPortal development process, leading up to, and during, the spring developers meeting. Some people in the community were trying to set directions.
Eric: Perhaps we should first do the uPortal 3 roadmap and up 2 sessions tomorrow, then after that look at the list from the spring and see how to revise it.
Susan: We need to know what got done and what didn't.
Jonathan: Integrate the work that Unicon has done on the 3.0 and 3.x roadmap into this process, and we need more discussion on that at this meeting. There needs to be another level above the wiki that will be a more accessible summary for institutional stakeholders etc. who want to know where it's going.
Eric: Jason, please make a list as the meeting continues of what is going on and then we will go through it tomorrow.
uPortal 3 roadmap - John Lewis, Unicon
We've divided it up into short range, medium, and long rage.
Short range - that could be used in pilot projects, and in limited production (limited in terms of features). Minimal IChannel support, only a few portlets, only the simple layout. The times in the spreadsheet are programmer hours, not including requirements/design.
Jim F.: This spreadsheet says 3/4 person year to get all the short-term stuff done. Which (in the given time scope) would take 6 full time developers.
Jonathan: Unicon plans to do the 3.0 base release. We will need other resources to continue to medium and long range goals.
Jim F.: 6 people gets you into the park of communications problems. As a user would I be willing to believe this number? The feasibility of achieving the number?
John Lewis: We already have four FTE's from beginning of month, plus whatever Peter can spare.
Jonathan: For December, we won't be able to rely on community, and the question will be, what do we need to drop.
Jason: What is the target audience for this?
John L: It should be usable for new adopters with small UI requirements. For other places, to give them something to look at and work on and comment about.
medium range: Get all the user experience parts in, DLM, Ajax responsiveness, more standard portlets; and performance testing.
long range: New rendering contexts for accessibility and mobiles, more administrative user interfaces, multiple organizations in one instance, documenting clustered configurations.
Are there things that are missing?
Are there things in the wrong timeframe?
It needs to be broken down into smaller releases. 3.0 for short, but we shouldn't wait for EVERYTHING in medium to be done before 3.1.
Sanity-checking level of effort estimates.
Bill: It's great to see the roadmap on the wiki.
John L: Some of the listed hours have already been burned off as we speak. We think we can make it.
Bill: Why not chunk them up into 1200 hour packages. There is no mention in the chart of non-functional requirements such as performance testing; should not be left until after everything is done.
John L: (the estimates do not include any testing or documentation either.)
BREAK FOR LUNCH
uPortal 3 status - Peter Kharchenko, Unicon
Bug fixes
Servant Portlets
- refactored to use the same rendering and URL construction/processing as regular portlets. Better aligned with JSR-286.
Registries
- consolidated registry logic in generic classes
Concurrency (multiple requests for the same user, e.g. if user is downloading a big file, should be able to interact with portal in another window.) They would like people to submit use cases, examples of situations where this is useful, otherwise they will not do further changes on this.
ACEGI update (to 1.0.1)
Portlet preference id generation
- DbLoader vs. DB-sequence id generation (these were generated both ways and the i.d. numbers were clashing)
New features
Back button support (1.0)
- Encode portal navigational state in every URL
- Navigation state in uPortal2 stylesheets
- Active tab
- Focused portlet
example
http://host/uP/a16/root/render.uP
where a16 is example of active tab, and 'root' is example of the focused portlet (i.e. at the moment no individual channel is focused).
Implementation
- AttributeDecoratorAndProcessor
- Configured with a list of rendering attribute names
- Configured for uP2 context (tabIdDecoratorAndProcessor)
Adding structured content from XSLT
Adding a new tab
- structure stylesheet may impose its own requirements
- for instance in uP2
- -- portlet can only be added to a column
- -- hence one would want to generate URLs that creates a Tab with columns
Solution
A general set of commands for adding more than one layout node
(Peter gave an example of what the XSLT would look like to do this.)
Jim F: question to Peter, the 'simple URL' suggested at the MIT developer's meeting, has that gone away?
Peter: No. Here, the navigation has been done with internal tab or column i.d. rather than a symbolic identifier.
uPortal 3 user experience - Gary Thompson, Unicon
(a designer's perspective. Context: there was a complaint that design suggestions had been made to the developers and nothing had come of it. So Gary wanted to explain what a web designer does and the value of it.)
Also see the User Interface page of the uPortal 3 wiki
(Mike Oltz points out that much of this is things he just typed from Gary's slides, until we get to the discussion part)
common understanding of design
designer's role in the community
speak to up3 work
action items
User experience: The sum experience of a user while interacting with the system or product.
The aim of design is to create good user experiences
Designers think about
- People and their goals
- How to adapt technology to people
- -- Attraction
- -- Satisfaction
- -- Representation
- -- Adoption
Common understanding of design
Art
artificial
no constraints
expression
Design
For use
has constraints
science
Design, while exploratory and often visual, is a scientific process with a commercial goal where the end product is used.
Design is science
observation
hypothesis
test & validate
Designers don't instinctively know the answer either - but we'll find out.
Designer's role in the community
What should a portal for higher ed be?
- the answer is pursued by the community
How should it be used by people?
- the answer is pursued by the Designers
How can it be done with technology?
- the answer is pursued by the Developers
Where is design in the uPortal community?
until there is more designer involvement and emphasis on making the product desirable and usable, uPortal will not reach its full potential
- powerful but unattractive functionality that is difficult to use
Significant barriers to entry for both product use and community contribution
Designers are not developers (usually)
speak a different language
have different goals
follow a different core process
use different tools
produce different artifacts
Designers are scared away when:
forced to learn and speak 'tech talk'
asked to design in a "dev environment"
misunderstood as "skinners" (we made something cool, now make it beautiful)
user experience is not prioritized
not given development support
It's symbiotic
- developers need designers to envision and design a good user experience
- designers need developers to build the experience
Collier: A lot of schools don't even have a designer on staff. What are examples of open source projects with user interfaces?
User_Centered Design (UCD) approach
- The chief difference from other interface design philosophies is that user-centered design tries to optimize the user interface around how people can, want, or need to work, rather than forcing the users to change how they work to accommodate the system or function.
UCD:
User Research - Observation
Analysis - Observation
Design Concepts - Hypothesis
User/Usability Testing - Test/Validate
He gave an example of: statistics about users of Yahoo from a report they found.
A design tool called a 'persona'. Fictitious characters created to represent the different user types.
You want no more than about half a dozen personas per user group. This helps the designer think about meeting the needs of each of the personas.
Eric: As a software developer, the persona approach sounds very valid. We have a much broader target audience, i.e. we have to meet their specific needs. Yahoo serves everybody in a general way, whereas colleges have legal obligations to include various user audiences.
Collier: The stuff JA-SIG does can only go so far, then a particular school will modify closer to their own needs. We don't want to design too far, so that a school has to undo some of that work to better meet what they want.
Jan: Laying this design process out for the developers so they can take the general concept of design as a consideration to their work, is the most important thing that developers can get out of this presentation.
next step:
Comparative/Competitive Analysis
(strong user interaction patterns - what are users used to doing that is similar to the application you are designing?)
Where are the patterns? They are in the User Interface part of the up3 wiki.
Heuristic Analysis - a set of industry-defined principles that govern good design
Gary gave an example of doing such an analysis to the admin channels of uPortal. (see Analysis in right column of such)
Susan: What our admin designs are reflecting, is a thin veneer on the code underneath. So it's developer centric.
Gary: It's not necessarily bad, if a developer is the primary user.
When presenting uPortal to admins or techies, often the admin channels are the thing most often shown ("and look what you can do under the hood") so we should expend some design effort.
Gary made an analogy with a car show - uPortal has a great engine, but the outside and dashboard are not flashy enough for people to look at it.
BREAK
Jan: A question during the break - we see that we have an impedance mismatch between the developer and design communities. How do we capture that mismatch and fix it over time? How do we fix these pain points? We have to document these. Maybe we make bug reports, maybe a set of wiki pages.
Jason: We don't have designers as such already, but there are people who are designish developers. It's more a task of linking up like-minded people.
Gary: The way that I see it, there are three roles: the designer; the Java developers; the web developers.
Jason: The documents you produced are exactly the kind of things we envision coming out of the requirements process. How do we tie these efforts together? Maybe the Java community is a very architecture driven community.
Gary: It was not immediately clear to me how I was supposed to participate.
Dan: I think this is in part a matter of personnel; I don't think we've had a "component owner" who is committed to UI issues and pursuing the matter, since Justin at IM&M no longer participated.
Gary: In open source, the work is done by someone who sees a need and chooses to fulfill it. Most of the designers who participate will not be in a position to build it.
Jan: In the I.T. personas there was no "Debbie Designer" listed who was going to do such things.
Andrew: The personas are to model the customers of our open source project. Does the typical school really have a designer to bring to bear on this issue?
Gary: In most cases, no there is not such a person in the portal group at a school. Collier is an exception (used to be Virginia Tech, now at UBMC or is that UMBC). What the schools want is, how do they reskin the portal for their site.
Jason: In many cases a piece of the portal was started at one site and then pushed out for others to use. In this situation it may be better to start it out in the public forum and get feedback earlier on.
Gary: We probably want to put out some standards and guidelines. How does it fit together as a whole? This gives a better sense that the parts of the application were designed together and are meant to be used together.
Jason: We tend to focus on the infrastructure usability. Do we need some guideline that applies to portlets as well? Our portlets are going to be used as examples by people developing other ones for the portal, so we should put the effort into their design as well.
(back to Gary's slides)
User Experience Goals
modernize the user experience
layout customization
improve usability
resulting in powerful and attractive functionality that is easy to use
(see 'UE goals' from the uPortal 3 User Interface page) Gary talked through this page.
Jonathan: This overall topic is something that will greatly benefit the uPortal.
Collier: We should have a 'uPortal installation wizard' (Mike Oltz's wording of what he was trying to say) like Confluence has, to ask the sysadmin what is the database, what is the LDAP, etc. and it fills in the config files for you.
Jim H. Don't eliminate the config files because it is worthwhile at times to edit them directly, and they are a good reference.
(back to gary)
Look at the uPortal design concepts documents from the User Interface page for some examples.]
Gary talked through the examples given in these documents. Many of the user options are hidden in popup menus that are shown only when the user wants to see them. In one example there is an 'immutable column' at the left. i.e. it is always there no matter what else you are doing.
User Testing
Effectiveness
Efficiency
Satisfaction
In the beginning, try the paper prototypes: that's a way to test it way early in the process. Then later, you can use onscreen prototypes and ask similar questions.
Jonathan: We have interest from several quarters who want to help out with usability testing, and have experience with doing this..
Support good design in the uPortal community
Prioritize work that results in UE improvements
Support UE improvements on the backend
Review design activity and give feedback
Get designers and UE people at your institution to participate
Incorporate design in your projects
Get users involved (you have direct access)
Submit design ideas
Do "Brick Wall" user testing of designs with users at your institution and post the feedback
Sponsor "Compliant" user testing at your institution
Jonathan: I presented a "fairly sobering account" with where we stand with respect to community attitudes about uPortal. What I don't want to happen, I don't want to communicate excessive negativity; I feel really positive that we can turn this around. We have a lot of resources that we can and will apply to move forward.
It's important that there be a session where Unicon gets the information on how to confidently move toward the November release; helping them to prioritize what to do.
John: Do it at the end of the day tomorrow?
TUESDAY
Less Hacking, More Configuring - Jan Nielsen, SunGard Higher Education
abstract: Jan "would like our collective contributions to be more cognizant of the support issue associated with changing method names and parms: try to deprecate old method signatures in favor of signature changes; and try to avoid deprecation in favor changes which don't affect backwards compatibility at all."
Jan would like to download the uPortal binary and be able to add the Luminis value-added changes without having to change the JA-SIG code base. This requires more care when making changes to the public API's, and when making changes to the database tables or the static portion of the content therein. He gave examples of the problem and more careful solutions.
Jan believes that using Spring will alleviate some of the problem.
Jan wants to make the properties manager into a pluggable provider.
Bill: I like the idea. It's a concern of Rutgers also. Many larger sites have issues with upgrading and moving changes forward. We've tried to minimize this, by making local improvements shared and try to get them into the main source tree. One thing that makes the compatibility issue difficult; there does not seem to be any clear definition of what are 'published' API's and what merely happen to be (haphazardly) public.
Jan would like to see a much cleaner framework representation (as per Bill) even for up2.x.
Chris: The portlet spec does this very carefully. A portlet can't call just any public method.
Jason: Jan are you talking more about an API for framework extension?
Jan: Yes it would help us to know that. I think a degree of refactoring needs to be done. Be more judicious about changes and document binary breakages explicitly.
Andrew: There are a couple common techniques in this space. The separate package approach, etc. We don't want to declare every public method as our official API. There are a couple common techniques to decide this. I think Eclipse does this by a parallel package structure, the officially public ones and a parallel one that actually does the work. We could also be documenting this more carefully in the javadocs. We do expose more classes than the ones needed for channel implementation that are nevertheless interesting (group implementations, etc.).
Eric: I do like the simplicity of separating the packages into jars, one of which is public and other(s) not. But this would be a lot of work to get there.
Jan: As in Apache projects, we would then have for example a uportal-api.jar and a uportal.jar.
Peter: How do we address this issue in up3? It would be nice if we had some procedure or criteria for deciding whether a class is public or private.
Bill: One thing that has worked out well for us in CAS 3.0 is to be very conservative with the internal implementation and try to hide stuff as much as possible. Scott (primary developer on CAS 3.0) has been religious about using 'final' everywhere possible.
Jason: Are we ahead of ourselves? Do we have an architectural guideline first, for future development? Then we can talk about refactoring existing things.
Peter: I think we should sort out what we currently have first, and that will help us develop the guidelines.
Jan: I like Bill's differentiation between public/private published/unpublished (Mike Oltz apologizes missing said remarks by Bill). I think that being conservative and leaving everything private until a specific request is made. We also use 'final' as much as possible.
Bill: Suggestion - a way forward, Jan, why don't you write up your refactoring proposal and that would be something to start from.
Jan: I could put such a thing on the wiki.
Chris: If we could start with a uportal-api.jar, where would we start with that, 2.6? or 2.5 patches?
Jan: We would encourage use of that mechanism as soon as possible.
Jason: This is more like development than a fix, and ought to be in the HEAD. We should make changes in the HEAD and port them into patches if desired, rather than making changes in patches and porting it into HEAD.
Andrew: This is something new we've never had before, formally published API's. This should happen somewhere where it would do somebody any good. But people who are upgrading to 2.5 are already going through the pain and it wouldn't undo the pain they've already been through.
(there was some more debate over Chris's question)
Jan: Discussion will continue on mailing list and the wiki.
uPortal 2.5 status - Andrew Petro, Unicon
Andrew is planning to work on cutting 2.5.3 (either RC2 or GA) Wednesday morning.
The big change is a fix to DLM and some other bug fixes.
So, where is 2.5.x going? Is it time yet to break API's?
Susan: When I make minor changes and bug fixes I have been putting them up as patches because it is the more polite course to proceed, rather than committing them. Are we supposed to go right away to commit them?
Andrew: Patch posting is a good way to communicate and do preliminary testing.
Susan: Should I say 'these four patches look good I think I will commit them' or just go and do it.
Andrew: communication is good, but if people are aware of what's going on and the Jira shows that people have tried it, then go ahead.
This is not working as well, 2.x releases are not coming out as often, it would be better to do it more often. Is anyone else interested in being the release manager of this branch?
Eric: It shouldn't be "well there've been 6 months so we should put out another release" But, how much change has there been. Who is going to do the change? People who are bothered by the bug. With the patches, we need to get away from the other extreme, 'we shouldn't release until all the bugs are fixed' There needs to be a middle ground, of doing the release when it feels necessary.
Jason: In Mozilla when they get close to a release they concentrate on what are the release blockers and who should do them. We (uPortal) are not using the Jira priority settings as much as we should. Even one patch, if it is crucial, would warrant a release.
Jonathan Markow: Are you looking for someone to take over for you? Or is it kind of optional?
Andrew: I don't feel good about it taking so long, so perhaps someone else could put more time into it.
a) We could keep doing 2.5.x patch releases
b) We could put changes in the head and have a 2.6
c) We could focus on uP3.
They're not mutually exclusive. Why not settle it now for 2.5.x.
Take the question of who will do it, to the dev list.
(one particular person was suggested)
Susan: It takes a lot of work to support uPortal on one's own campus and also look at a base version and test the patches to that.
Jonathan: Let's ask that particular person and see how it goes.
Jason: To sweeten the pot to get someone to do that, say "I, Andrew, could work on certain bug fixes I have in mind, if I weren't release engineer". Or, why don't you just announce that you are leaving it, rather than being indefinite, to put more urgency on someone to step up.
Jan: Can we get something on the wiki about the responsibilities of a release manager, e.g. what Rutgers did on 2.4, to clarify what the new person would be stepping up to?
Bill: I support documenting the roles further.
Chris: What about the Xalan issue raised recently e.g. by Faisan?
Andrew: We have reverted out the change Faisan made and document the problem.
Chris: In the past we have been aggressive about fixing security and memory leaks, and this is a memory leak which is potentially fixed.
Andrew: Large heavy memory usage sites can go to the newer version of the jar. But using the new one introduced a bug that some XSLT's with empty elements of the form <page/> would make NPE's. Faisan thinks he has tracked down that bug in Xalan and fixed it, but it has not been well tested yet. So what we are doing in 2.5.3 is to not address the issue yet. There is "more tapdance than we want to do" before tomorrow morning.
Some people's comments indicate that the Xalan developers won't accept Faisan's patch, that there aren't supposed to be such elements.
Dan M: We should not release uPortal with this known memory bug.
Susan: Does this affect any bundled channels? There was an issue with GAP that got patched. The problems have not been tested well enough.
Bill: I think we should go out with this fix in uPortal 2.6.
Dan M: I disagree.
Susan: Should we do it at 2.5.4?
Bill: Given the release strategy, this causes pain.
Andrew: Since nobody has taken over, I get to call this. I want to cut 2.5.3 really soon, so we are going to leave this the old way. Let's concentrate on addressing this in 2.5.4, we will have more time to evaluate that. But I think this one issue is too small to go to 2.6.
Chris: I don't think we should be putting out any releases with a known memory leak; I think it's irresponsible.
Eric: I agree with this. Security and core stability are crucial issues. I don't think we should refrain from fixing this in order not to break something else.
Bill: Let's ask this quickly on the dev list.
MORNING BREAK
Hal: (a guy at UC Riverside) who was videoconferenced in (not Nassar) asked about greater flexibility in how channels would be laid out in a layout. people at their site wanted more of a content management layout rather than a channel model.
Gary: (the designer guy) This ought to be possible, there is a recommendation for having a layout grid such as I mentioned in passing in my talk.
Jason: (to UC Riverside) You could do some of this by having extra channel properties and then modifying the theme xsl. Why don't you post your mockups, that you don't know how to do, on the list or the wiki.
uPortal 2.6 update - Jason Shao, Rutgers
The gist: we as a community don't really have a clear conception yet of what 2.6 actually means. SunGard was hoping to contribute several things. There should be a discussion over what is going to be going in.
Chris: Elimination of IChannel, some of that work is in HEAD.
Andrew: I anticipate that some of these Academus changes may not fit into the release strategy for a patches branch, then they would go into the HEAD.
Jan: From Luminis platform, we want to contribute some internationalization, configuration enhancements, improvements in DLM.
Andrew: Doing something about the public API issue. Also, tasks toward the upgrade path to up3. Curtailing the use of API's that won't be in up3. Translating a channel to a portlet probably isn't hard, but it may do some custom stuff under the hood that won't go forward. JSR-168 support issues beyond that.
Jason: The work that Peter's done for IChannel support in up3 may be a good starting point for the public API definition.
Peter: The compatibility page lists what classes are included, some of them are sensible, and some should not be necessary. e.g. we can load things with CAR, and that requires a lot of other classes that we may be able to refactor out of it.
Susan: What are the enticements to go to 2.6? It would be nice if some of Gary's suggestions were there, but if it's a better DLM then that's enough. Is there something that could be done in 2.6 to move toward better designs?
Jason: up3 will be supporting the up2 rendering context at first, so could we get some of that in 2?
John Lewis: There are limitations in up2 that make some of their ideas tough to implement in that, for example, Ajax interactivity; it will be much easier in up3.
Peter: I concur.
Jason: Okay, besides the public API, what are some other things we can do?
Peter: We could go through the detailed list of 3.x work, and decide which are possible for 2.x. This is not going to be an easy question to answer.
Jason: We want to do statistics gathering and monitoring, so if we do that locally that might be a good example of what up3 features we would like in 2.x.
Jan: What about Spring framework integration? Are there pieces we can pull back into 2.x?
Peter: Maybe we could do that for DLM, but there's not much code at present we want to carry over for up3.
Andrew: Get very serious and aggressive about using Spring? (but who would be available to do the work) What are the most painful points in uPortal? The PortalSessionManager. How real is the potential for going in the other direction? Are there bits we could backport, to get guinea pigs out of the current user community?
Peter: Well the UserInstance class, perhaps the one from up3 could be moved back but it would be an awful lot of work.
Peter: Preparing ourselves for the future - perhaps let's look at moving some bundled channels over to portlets.
Jonathan: From now on, any change to 2.x should at least ask, does it ease migration?
Andrew: I agree about moving channels to portlets. All these efforts will just drop into up3.
Dan M: I thought the reason for going to 3.0 is that 2.x can't support all features of portlets?
Jason: Is this the time to migrate layouts from SLM or ALM to DLM and deprecate ALM in 2.6? And if we wrote such a migrator, would we then have to migrate again from up2 DLM to up3 DLM?
Peter: The 2.x DLM is based on the SLM format, and we want to refactor that and the table structure will be different, so we will have to migrate it again.
Andrew: Should 2.6 be the release where we go after the import/export features we wanted? Having an admin channel that can dump out layouts as a massive messy XML file, and then be able to read it in wherever.
Susan: It would be better to have import/export tools now, and shake out the bugs now, and we can then enhance them for the 3.0 transition.
Eric: The consensus seems to be we should get people moved to 2.6 and point that toward 3.0. We need a release manager!
Jonathan: We need to communicate to people that it is a path to 3.0.
Jan: I will ask at SunGard whether we could do it, since we are planning to contribute a lot to it. No commitment at this time.
Eric: I think we are piling too much on the release manager, it's not their responsibility to be sure that issues are resolved. All they have to do is make sure that people who ARE doing work, check it in and that it is tested, then build it. If someone is being bothered by a bug, then the bug fixing needs to happen but it's not the release manager's responsibility to go out and get someone to do it.
Jonathan: The release manager should coordinate and delegate work.
GAP Status - Dan Ellentuck, Columbia
1. The work on PAGS (PersonAttributeGroupStore) is a crucial point.
2. Using John's task list to short and medium GAP tasks.
PAGS - for more than 6 months hoping that Mark Boyd and Keith Stacks were going to do the PAGS migration. They had articulated some permissions enhancements, many of which could be addressed in PAGS, so it felt like a natural fit. But they weren't able to undertake that. When Dan thought about it, it turned out to be a more significant piece of work, a) it would be nice to do those enhancements, and b) once split off from uPortal, PAGS is a different animal, no more PersonDirectory service and no concept of a person. Instead it is now an "entity attribute service" collected from a variety of sources. This complicates the design but also enriches it. Multiple-typedness and multiple sources of information. Mark and Keith envisioned a much richer grammar for describing attributes.
Short-term delivery for up3. The GAP stuff has to be delivered by early October if it can be usable for a November release. Getting the PAGS service integrated within GAP, and completing the initial release of the GAP module. Addressing the issues with the GAP manager portlets is out of scope for Dan in that time frame.
Medium-term goals for up3. (a) Migration of the JitLDAP service. The author of that is no longer supporting their uPortal and was not aware of anyone else who could pick it up. (b) The backend store that describes the group structure of PAGS is available through a DAO; the October one will use the legacy, up2 syntax of the XML stuff. For medium-term he would like a db-based DAO description so that it could be updated dynamically. (c) Creation of a nightly build process.
John L.: Sounds good re up3. We appreciate what you're doing.
Jonathan: Do we want Jan to take back the question of whether Mark and Keith can get involved?
Jan: Our product managers have required us not to contribute that to the community. That's not going to happen any near time.
Perhaps communication from the JA-SIG board or from large sites would help.
AFTERNOON BREAK
What have we been working on and what is happening in next six months.
Rutgers: (Dan M.) Moved to 2.5+ in June and in production for 3 months, added some performance improvements. Created a content cache for the Proxy and XSLT channels. We got hit really bad for fall rush, we ran on 3 servers instead of 4 (no choice) and we had like 70,000 logins in one day. Needed to add more instances of Apache to handle it. (Jason) we have a MessageOfTheDay kind of channel now. From the spring task list, XTHML theme and CSS style sheets are in the HEAD, and such has been configured to use DML as a default. We haven't had much progress on SLM to DLM migration. Targeted for local deployment next summer. (Dan M.) Preparing for doing benchmarking.
Dan E: Most of the work recently has been in rethinking of PAGS more of an entity attribute rather than a person attribute service. Making it more configurable and a richer grammar. We have a build already that's workable with up3, and all these things I'm talking about would be enhancements to that.
Jim H: Eric has done a lot to improve U. Wisc's own portal, and now he's going to be doing half-time community work for the rest of this calendar year. (Jim is Eric's supervisor)
Eric: I haven't done much over the last six months due to changing jobs and other changes. I will be working on uPortal 3 and helping with the RC and GA of it.
Jason: Has the PersonDirectory externalization been completed? (now in a separate jar?)
Eric: Yes but we haven't documented it; we can distribute it soon.
Dan E: Releasing just source, or the jar?