An Evaluation Strategy is crucial to the success of the project. The purpose of the evaluation is to guide the project's development and project planning processes, to assess and improve the design of the INHALE/INFORMS materials and to share the project's learning with stakeholders. Each stage of the evaluation process will be documented and include reliable data collection and analysis. This evaluation plan is based on guidelines for evaluation produced by the Tavistock Institute^[1] and applied evaluation techniques described by the HyLiFe Project^[2]).

The evaluation priorities are as follows:

• Development of impact factors to ascertain the effect of the materials on users and staff skills and awareness of IE resources
   
• User feedback
   
• Take-up and use of the different materials within modules in the VLEs
   
• Investigation of cultural change within the partner institutions
   
• Monitoring and reporting the progress of the Project including information for stakeholders

There are five main phases of evaluation.

Formative evaluation:

1. skills audit of selected students and gathering of baseline information from selected users and librarians
    
2. prototyping of INHALE/INFORMS interface with small groups
    
3. embedding of the materials within LEARN and Bodington VLE's
    
4. testing the Learning Object Database and other project tools at partner institutions
    
5. testing the evaluation methodology at partner institutions

Project milestones will trigger presentation of findings including:

Summative evaluation:

1. reporting and evidence presentation
    
2. exit strategy and business plan
    
3. cultural changes within the partner institutions in relation to VLE's
    
4. changes in user behaviour
    
5. technical report including standards and interoperability issues

Success criteria and related performance indicators, will be integrated into the project and guide the design of evaluation tools. The table below demonstrates this process.

The INFORMS project team will work closely with the JISC DiVLE LINKER Evaluation Project.

The INFORMS team will be supported by a Steering Group which will meet at least twice during the lifetime of the project. The Huddersfield team will have fortnightly meetings and be in regular contact with the two partners by email and telephone, as well as undertaking as many full team meetings and workshops with them as time and distance will allow. The systematic recording of project learning will underpin the entire process.

A variety of quantitative and qualitative evaluation methods will be used - the INHALE evaluation tools will be written into the model so that potential user communities will have the necessary evaluation mechanisms for their local clientele. A final evaluation stage will draw together the project outcomes and be a major element of the Project Report in 2003.

Data collection, recording and analysis

For each of the evaluation priorities appropriate techniques will be designed as shown below:

Development of impact factors to ascertain the effect of the materials on users and staff skills and awareness of IE resources:

User feedback:

Take-up and use of the different materials within modules in the VLE's:

Investigation of cultural change within the partner institutions:

Monitoring the progress of the Project including information for stakeholders:

The field testing of the materials must be as rigorous as possible. The target population for the baseline survey at Oxford University is Clinical Medical, English and Social Science students and staff and at Loughborough University it is Business, Sport and Exercise Sciences and Engineering students and staff. The range of schools and departments within the two partner sites and various student "types" and skill levels will be used to test generalisable findings. Triangulation with Huddersfield results will validate this process.

For each evaluation tool a proforma will be written which outlines the information required, the data to be collected, how it will be analysed and presented. Examples for the Baseline Survey, the Pre-test and Post-test surveys are in the Appendix. As with other elements of the project, evaluation mechanisms will be amended in the light of practice, to ensure that a systematic and structured approach is maintained throughout. Results will be anonymised to encourage participation and openness.

No ethical issues are foreseen, the identity of participants will be kept confidential with full informed consent and withdrawal allowed at any stage. At Loughborough permission to conduct the project has been given by the Pro Vice Chancellor, Deans or Associate Deans and also head of Departments and academics of the involved schools and therefore no polictical issues are foreseen. At Oxford no issues are foreseen. Throughout the project the teams will be sensitive to the needs of academic staff and students. We are aware that participation in the piloting of the INHALE/INFORMS materials is voluntary and involvement must not place students at a disadvantage in terms of time spent or compromise achievement of learning outcomes. For this reason it is not envisaged that the materials will be used in summative assessment activities. However this issue will be kept under constant review during the project.

Full informed consent from students will be obtained with withdrawal allowed at any point. All data obtained will be kept confidential and annoymised in reports.

Copyright requirements and IPR related to information sources used by the materials will be strictly adhered to.

Access at partner institutions will be carefully considered. At Loughborough no access problems are foreseen, students have access to computers in departments, open access labs and work stations in the Library and all academic staff have access to their own computers. At Oxford no access problems are foreseen, there is a library training centre, where learning support and activities are monitored and liaison is already in place with the medical School in providing a search workshop there to students on their Foundation course each year. Trials of the materials at Loughborough will mainly take place in timetabled academic sessions. Trials at Oxford will take place either in the Library or as part of weekly turorials, under the guidance of a member of Faculty.

A dedicated Evaluation Officer is a member of the Huddersfield Team and will use the expertise and experience developed during the evaluation of the INHALE Project to evaluate the data gathered on the INFORMS Project. The Evaluation Time Line is shown in Appendix 4.

As shown in the evaluation model and recording phases above, reporting and evidence presentation will be carried out throughout the project. This will take the form of various written reports and dissemination - notably:

• Triannual Reports to the JISC Programme Office
   
• Progress Reports for the Project Steering Group
   
• Evaluation Reports for each pilot module
   
• Exit strategy and business plan
   
• Technical Report including standards and interoperability issues
   
• Articles in the literature and conference papers
   
• INFORMS News items on INFORMS web site