next up previous contents
Next: Combining the scores and Up: Technology Evaluation Report and Previous: Method   Contents

Score Justification

Criterion A

All the platforms tested are domain agnostic before specialisation; it is the tools that make them domain specific. They all score 50.

Criterion B

Sakai: a) In its current form Sakai is primarily an e-Collaboration framework, in that there are facilities for online collaboration pre-bundled. There is also currently a software patch that allows Sakai to pick up X.509 certificates from MyProxy servers for Grid computing use. Sakai thus scores 60 for the e-Science sub criterion.

b) Sakai can also be used for e-Learning, as CHEF is at U. Michigan. It doesn't yet offer any specialized tools for things like assessment and IMS package delivery, but it does offer a working paradigm that lends itself well to representing courses and the students on those courses. Tutorial sessions can be held online, using the bundled collaboration tools, and course materials can be presented in the form of the multiple web pages that you can assign to each course worksite. Sakai scores 50 in the e-Learning category.

CHEF/ OGCE: a) The OGCE framework, based on a populated version of CHEF, is the clear winner in the e-Science category. OGCE makes use of CHEFS e-collaboration tools and adds grid computing tool components. NEESGrid in the USA use it in their earthquake simulation experiments. OGCE scores 70 in the e-Science category.

b) CHEF/ OGCE scores slightly less then Sakai in the e-Learning category as it offers similar facilities. See the Sakai section for more details. CHEF/ OGCE scores 45 here.

Bodington: a) Bodington is primarily of use as a learning environment. It has strong content management facilities and has tools for generating and displaying assessments complying with IMS QTI (Question and Test Interchange) format. It is, however, not of as great use to the e-Science community, who need collaboration tools and user grouping and administration features more than content navigation. Bodington scores 30 here.

b) This is where Bodington is more fitting. Bodington is a content manager by design, it uses a library metaphor to arrange content into floors and rooms; this is obviously a successful approach as Bodington has been in production at Leeds for around 4 years. A drawback of the Bodington approach is in the centralization of content as opposed to a strategy of organizing metadata on remote content. Bodington scores 70 here.

GridSphere: a) GridSphere is a portal framework in the sense that it is an unspecialized container for components that, when added will increase its utility within a specified domain. The main take-up of GridSphere has been within the hard science community and several projects have successfully added portlet components to GridSphere and put it into production. At the GridSphere website you can download a collection of Grid portlets for accessing Globus functionality. GridSphere scores 70 here due to its good Globus Grid tool support.

b) GridSphere scores poorly in the e-Learning sub criterion. There are no tools bundled, or in the pipeline, that are applicable to this domain. GridSphere is a framework designed for specialization however, so the potential is there to add e-portfolio, course management and assessment tools. GridSphere scores 20.

Criterion C

Sakai: a) Sakai makes good use of open source libraries from other projects. It uses the Spring framework, Hibernate, Java Server Faces, Pluto and Velocity amongst many others. It is also re-using lots of the CHEF tool code, which has been tested in the live environment of U. Michigan. For these reasons Sakai scores well and gets 70.

b) Sakai uses the OKI OSIDs for component messaging internally, although these interfaces are not exposed to external tools. There is no support for WSRP or JSR 168 tools presently, JSR 168 support has been in the pipeline for some time and WSRP is being worked into the architecture design at the time of writing. In reality, the work on formalizing the software interfaces used in Sakai is still ongoing

CHEF/ OGCE: a) CHEF uses open source libraries from other projects. It uses Struts, Velocity. JUnit and James, amongst others. CHEF scores 60. On b) CHEF scores 20.

Bodington: a) Bodington is a Web application written using servlets for its dynamic aspects. Bodington does not leverage any Web application development frameworks like Java Server Faces, Struts or Tapestry, primarily for the reason that the architecture was effectively formulated in 1998, before the advent of these frameworks. Bodington also doesn't make use of any software libraries that aren't included in the Tomcat installation. As this metric depends on the utilization of cross-project libraries, Bodington scores 30.

b) Bodington's modularity comes at the Java Servlet level. If you want a new tool, you write a servlet. Data repository (DR) services are provided, but are not, to the authors' knowledge, currently based on ratified standards. As with all the other frameworks discussed, you cannot aggregate either JSR 168 or WSRP compliant portlets, so tool creation has to be done by proprietary means. Bodington scores 20 here.

GridSphere: a) Gridsphere scores 80. On b) GridSphere scores 50.

Criterion D

All the frameworks considered score 100 here as they can be displayed in any CSS and JavaScript enabled Web browser.

Criterion E

Sakai: The Sakai project has a formalized community process in place called the SEPP (Sakai Educational Partners Program). The need for the SEPP was actually expressed by the main Sakai funders, the Mellon Foundation, at the Sakai project's inception, and as such is a pre-requisite for funding to continue. To join the SEPP, institutions have to pay a fee of $10K per year for three years. Now that version 1 has been publicly released, there are several public mailing lists for developers and users outside the SEPP. The mailing lists have been dedicated slight more to developers, so Sakai scores 70 and 60 respectively.

CHEF/ OGCE: The OGCE project has a good selection of majordomo mailing lists, and a large contact list on its Web site. OGCE scores 60 in each category.

Bodington: Bodington is a SourceForge hosted project and has a mailing list dedicated to developers. Traffic is relatively high and this reflects the list's utility. It would be useful if the lists were broken down into more specialized subject areas as there is a lot of material on them. Bodington scores 60 and 30 respectively.

GridSphere: GridSphere has both a user's and developers mailing list and so scores 60 in each criterion.

Criterion F

All the platform tested are presentation agnostic, so they all score 50.

Criterion G

Sakai: Sakai's documentation is patchy and still under development. Seeing as the architecture is still in flux, this is understandable. The state of the documentation is made up for to a degree by the quality of Sakai's community efforts. Sakai scores 20 in each sub criterion.

CHEF/ OGCE: OGCE's documentation is better, and more copious than Sakai's, as befits a longer running project. The fact that CHEF long since standardised on its architecture obviously helps. CHEF gets 50 in each category.

GridSphere: GridSphere's documentation scores 60 for both quality and quantity; there is a collection of documents in the form of ``howtos'', both on the Web site and in the binary distribution.

Bodington: Bodington scores 40 in each sub criterion. The Bodington gatehouse site hosts a collection of tutorials for students wishing to use the system but these are obviously targeted to its use as a VLE not a VRE.

Criterion H

CHEF/ OGCE is the clear winner here as it comes with all the collaboration tools of Sakai, plus a suite of useful Grid tools. Sakai comes second, as it comes supplied with a set of useful collaboration tools such as chat, threaded discussion and shared file space. GridSphere comes third as it comes with chat, file space and chart creation tools. Bodington comes last in this criterion, due to the fact that the out of the box toolset relevant to a VRE is sparse, as its intended role is that of an e-Learning/ content management environment, not a tool delivery framework.

Criterion I

Sakai: Sakai funding has only just got going, so we have given it 60 for each sub-criterion.

CHEF/ OGCE: CHEF is effectively undergoing a transformation into Sakai and it is unclear what the funding situation currently is. CHEF is still in production use at U. Michigan. CHEF gets 50,10,0 in each sub criterion respectively.

Bodington: Bodington is funded by individual project grants from funding bodies, it does not have a concrete funding strategy like Sakai. There is however considerable institutional commitment in the UK and this should really be considered in a funding assessment. As far as we are aware there are no Bodington projects funded past two years scores, so it scores 50, 50 and 20 respectively.

GridSphere: GridSphere funding is about to end. Furthermore, GridSphere is currently trying to get funding from Deutsche-Grid initiative to continue support for GridSphere, but now it looks like it may be only enough money to support only 1 developer for 3 years in the worst case, so we have only given them 20 in each sub-criterion.


next up previous contents
Next: Combining the scores and Up: Technology Evaluation Report and Previous: Method   Contents
Rob Allan 2005-05-09