This section of the toolkit aims to help you design your evaluation and begin to plan your approach. It covers the following six areas:
Understanding your evaluation aim ^
The first stage of planning or designing your evaluation will involve establishing what exactly it is you want to find out. In other words… you need to understand your evaluation aim or objectives.
The main aim of your evaluation is most likely to be to assess the impacts of the residential on your students. To achieve this, here are some examples of questions you might ask:
- What changes have occurred as a result of the residential?
- Who has benefited from the residential?
- How and in what ways have they benefited?
- What have your students learnt? Is this what was intended?
- What kinds of experiences have they had? How are these experiences meaningful to them?
- What do they think and feel about their experiences?
- Have their attitudes or behaviours changed?
- How could the residentials you offer be improved in future?
You will also want to start thinking about the types of evidence you will need to effectively address your questions or objectives as you progress. This will inform how you plan your evaluation.
A benchmark is a point of reference against which you can measure the changes that have occurred as a result of an activity or experience. This reference point is also described as a baseline i.e. the starting point used for comparisons. In our case, the purpose of a benchmark or baseline is to provide an information base against which to assess a residential’s impact and effectiveness both while it is happening and after it is completed.
Therefore, to truly evaluate the impact and effectiveness of a residential it may be necessary to collect some baseline data or evidence before the residential takes place. This could be qualitative or quantitative, or both. Suggestions for how to do this are outlined later.
Devising your methodology ^
Once you have your evaluation aims and objectives or questions defined, the next step is to devise your methodology.
This stage involves deciding the most appropriate way to meet your evaluation aim and objectives, and answer your evaluation questions.
To decide upon the most effective methodology, you might think about the following questions:
- What data do you think would be most valuable to collect and why?
- What sort of data can you collect?
- How much data do you expect to gather?
- What is your preferred method of working with data?
- What kind of data would be useful to show to others (e.g. other staff, senior leaders, governors, Ofsted, parents or carers and funders)?
- What kinds of statements do you want to make from your evaluation?
Ethical considerations ^
This may seem obvious, but it is important to gain the consent of all the participants involved in the evaluation. This could be done informally or formally and will depend to a certain extent on your school’s policies and practice, and the ways you ultimately intend to use the evaluation evidence.
You should make sure that you have given all the participants – students, school and other staff, and parents or carers – the information they need to understand exactly what you are planning to do (and possibly therefore to decide if they wish to participate). It might be appropriate to get written consent from the parents or carers of the students involved. Typically, you might design a consent form that would:
- Describe the evaluation, including details of why, when, where and how the evaluation will take place and how the data will be used.
- Tell participants, and if necessary their parents or carers, that their participation is voluntary, and can be withdrawn at any time.
- Allow participants to provide a signature of their informed consent.
Once you have collected data, you should ensure that any confidential or sensitive data is stored securely. When you write up your evaluation, it is good practice to make ‘named’ participants anonymous by, for example, using pseudonyms or labels (Student A, Year 6 parent, etc.).
You may also need to carry out a risk assessment for some evaluation activities. Your school will also have its own child safeguarding policy, which you might need to consult.
The difference between qualitative and quantitative data ^
Qualitative data is information gathered from what people say and feel and what is observed or deduced. Evaluation of learning requires a strong focus on qualitative evidence and quantitative data. Quantitative data is any information that can be counted.
When planning your evaluation, you will need to decide whether it is possible or preferable to use qualitative or quantitative methodological approaches, or both. If you are interested in or need to collect numerical data, this can be described as a quantitative approach. It will involve methods such as surveys, as well as using – for example – data about attainment, progress, attendance and behaviour. If you are interested in more in-depth data, you will probably want to use qualitative methodological approaches such as focus groups. Both of these approaches are described in more detail later.
You do not have to choose one approach. Your evaluation is likely to be more informative and powerful if you use a combination of different methods.
Making sense of your findings – analysis and writing up ^
There is no point undertaking evaluation if you are not going to use it in some way. Once you have gathered your evaluation data, you will then be in a position to:
- Analyse and make sense of the data you have collected in all its forms.
- Complete your evaluation report.
Later in the toolkit, as well as step-by-step guides and examples for a variety of data collection methods, you will also find some pointers for how you might go about analysing your data. How you complete your evaluation report will depend upon your intended audience. The aim of your analysis should be to try answer questions such as:
- To what extent were the residential’s aim and objectives achieved? How do you know? What evidence do you have?
- What impact did the residential have on the participants? How do you know? What evidence do you have?
- Were there any unexpected outcomes? What were they and why may they have occurred?
- What would you do differently next time?
Writing an evaluation report
You will probably decide to produce some kind of written report. This may in fact be required by funders (who have funded or could be approached for future funding) or be useful when providing detailed evidence of impact to others.
You may share your findings with your students, colleagues, governors and/or parents during assemblies, INSET sessions, at parents’ evenings or during meetings. A PowerPoint presentation may therefore be an effective way to produce your evaluation report, or summarise a fuller written report.
Briefing notes and posters/displays
Briefing notes are normally one or two-page documents and can be used to present the main findings. They are a snapshot of ideas, findings and recommendations and can be effective as they are more likely to be read! Briefing notes can also be presented in leaflet format. Posters or displays can also be effective for communicating evaluation findings in a more visual and eye-catching way, especially to students and parents.
Films, articles, website and social media
Your students are likely to be the most effective communicators of your findings. You might therefore consider producing a film, or audio recording, in which they communicate significant findings, impacts or ideas. You might also consider approaching local radio, TV or newspapers to share and celebrate your findings.
Think about making your findings available online on your school’s or a residential provider organisation’s website, for example, so that anyone can download them. You might also summarise findings in blogs or through other social media, such as Twitter and Facebook. Don’t forget that websites and social media can also be used to display any films, presentations or posters.
Use any or all of these approaches as a promotional tool to celebrate the benefits and outcomes of the residential, and to gather support for future residentials.