As we approach the time of year when projects are due for evaluation, I am pleased to share our views and experiences of evaluating a wide variety of projects. Based on recent enquiries, I thought it would be useful to frame this fact sheet in a question and answer format:

Q. What are the advantages of our carrying out an independent evaluation of our project? Surely we are the best placed to report what has been going on?

A. Whilst you will hold the majority of data and information about your project, others may feel it is in your interests to deliver a positive report, which may be seen as biased and lacking an objective view point. It can also be difficult to both praise and constructively advise those who are close to the project. An outsider can bring an objective view point, and should not shy away from telling you and others what you or they need to know. It is also often the case that you can be too close to your project, and can miss good practice and outcomes that are under your nose. A trained and experienced ‘fresh pair of eyes’ is always ideal in my opinion. It is also often the case that funders require an independent evaluation.

Q. Is it the role of an evaluator just to report on inputs, outputs and outcomes?

A. Whilst a key component of an independent evaluation is to report on inputs, outputs and outcomes, I feel that it is also an evaluator’s role to identify good practice and challenges faced by the project. Whilst good practice is important, we learn more when things go wrong, or when we face unforeseen challenges. It is my view that others, from outside your project should also learn from both, so we should not shy away from sharing both with others. I also feel that an experienced evaluator should have the skills and experience to help you move forward; making recommendations on how to overcome challenges, and how to influence future strategies. Experience is not just about carrying out evaluations. It is about having experience, based on actually doing the job, and having a history of working in a field relevant to the work that is being evaluated.

Q. Is evaluation just about questionnaires and telephone interviews?

A. Whilst questionnaires and telephone interviews are valuable and proven research tools, I feel it is important for an experienced evaluator to get to the ‘coal face’, and actually meet the people involved in the planning and delivery of the project. I favour structured one to one interviews and case studies. We have also gained a great deal from workshops, where delegates from partner agencies and members of local communities are provided with the tools to have their say and freely express themselves. We have also developed an ‘Interactive Reporting’ toolkit, which involves the use of video interviews, based on the ‘Diary Room’ in the ‘Big Brother House’. You will be surprised to learn how freely young and old express themselves when they are being filmed. I also gain great value from observations, when I become part of the team I am evaluating. Not surprisingly, I gain the most when they forget I am there.

Q. How much detail should be included in evaluation reports?

A. Evaluation reports should be a concise ‘business tool’ that answers all of the questions set out at the start of the commission. In addition to reporting on inputs, outputs and outcomes, they should report on ‘wider impacts’, which are often unforeseen and unexpected positive consequences of the project. Reports should also contain well thought out recommendations, which should assist the commissioner with future strategies, which can of course include exit and / or succession options. We also favour placing our reports on websites, so others can learn from projects; taking advantage of good practice and lessons learnt.

Q. What are the benefits of a good independent evaluation report?

A. A good report will:

• Satisfy you, funders, partner agency members and stake holders that you have delivered what you promised
• Assist in tackling challenges
• Assist in delivering value for money, and high quality projects and initiatives
• Direct future strategies, which can include exit and / or succession options
• Assist in sharing good practice and lessons learnt with others
• Positively market and promote your successes and achievements
• Attract others to support your work
• Support future funding applications

Q. What would you say is the most important part of the evaluation process?

A. Listening and learning

Q. How can I locate the best person to carry the evaluation of our project?

A. Recommendation is the best source in my view, so always speak to, or seek a reference from someone from a project that has been previously evaluated. Also ask to view and read previous work carried out by the evaluator. Always speak to the person who will be carrying out, or leading the evaluation. Beware of being ‘fobbed off’ with a junior, more inexperienced member of a team when the evaluation process progresses.

Q. Where can I learn more?

A. Please feel free to contact me at edwin@widerimpact.com , when I will undertake to get back to you within 24 hours.

Kindest regards

Edwin Lewis

Director