SECTOR: Project Support (SECTOR Grants)

Description of process

Evaluation is a systematic and objective assessment of the design, implementation and results of an ongoing or completed project.

Project elements which will be evaluated are:

purpose, or the major positive change to be effected by the project (short-term benefit);
goal, meaning the long-term benefit of implementing the project.
inputs, including the various resources applied to produce the intended results, or outputs
outputs, or the immediate results that the project should guarantee

The aim of the evaluation is to determine the impact, efficiency, effectiveness, sustainability and relevance of the project. Here is a short description of each:

Impact means the foreseen and unforeseen consequences of the project. Assessment relates to the goal and the purpose of the project and needs comprehensive investigation. Different impacts may occur at various times. Evaluation should consider both short-term and long-term effects. Evaluation of short-term impacts is often undertaken while the project is still in operation. Long-term impacts may be difficult to predict (e.g., an infrastructural project shows quicker impact than, for instance, an educational one).

Efficiency is a measure of the outputs of the project — qualitative and quantitative — in relation to the total resource input. It shows how economically the various inputs are converted into outputs. Efficiency is a measure of the “productivity” of the implementation process: i.e. to what degree the outputs achieved derive from efficient use of financial, human and material resources.

Effectiveness is a measure of whether the purpose of the project has been achieved, or, the degree to which the outputs contribute to achieving the intended purpose. It shows if the project is heading in the planned direction. Assessing effectiveness is part of the ongoing monitoring, as a basis for evaluation. In order guage whether objectives are being fulfilled, they need to be specific, measurable and time-bounded.

Relevance is an overall assessment of whether the project is in line with both the organisation’s and the donor’s priorities. This is intended to clarify if the project should be accepted, rejected or reformulated. Relevance is an issue of vital importance because it measures the degree to which project priorities match the broader priority context.

Sustainability is an overall assessment of the extent to which the positive changes due to the project are expected to last (durability of success). Sustainability is thus a higher level test of whether the project succeeded or not. Donors put increasing emphasis on the long-term perspective and lasting improvements. Sustainability concerns what happens after the project has been completed.

The REC’s ongoing evaluation of the project results is part of the monitoring process and will serve the purpose of:

improving the quality of the project content and expectations; and
improve the REC’s knowledge and understanding of the issues addressed by the project.

Additionally the final result evaluation will serve the purpose of documenting and promoting the grants results to the donor, political authorities and the general public.

Evaluation of the grants results is done thorough:

a mid-term review, which is carried out within the project implementation to verify the relevance, sustainability and efficiency of the project and recommend adjustments to the grantees;
end-of-project evaluation, which assesses the lasting impact and sustainability of the project results as well as its effectiveness;
post-project evaluation, which checks the sustainability of the project results after the grant is closed.

At the REC the two first assessments are conducted systematically as part of the grants management process and the third one is subject to funding availability.

The final grants evaluation process at the REC includes the following steps:
1. NGO self-evaluation
Grants evaluation at the REC starts with the self-evaluation of the project results by the NGO as part of their final report. Through the grants reporting instruction document, NGOs are encouraged to describe the project implementation analyse the project and its impact with the help of indicators.
For this reason NGOs are required to make an analysis and evaluation of the project implementing organisations regarding:

the extent to which the objectives of the project were met and how they contributed to the overall goal/scope of the call;
the evaluation of the target groups/ beneficiaries of the project and its activities and results;
the sustainability of the project results;
overall conclusions about project;
the extent to which the project can be replicated; and
the potential or planned follow-up (new initiatives which were generated by the project in each of the countries involved).

2. REC evaluation of the project results
The REC grants manager assesses the project’s impact based on

on-going assessment of results and feedback through grants monitoring; and
sssessment of the grants reports.

Overall criteria for the evaluation of the project results includes:

Impacts, including long-term positive and negative effects of the project on the environment, beneficiary countries and target groups according to indicators. Impact assessment involves:

assessment of the extent to which the objectives of the project were met and how they contributed to the overall goal and scope of the grants programme.
getting feedback on the project’s reception by target groups and beneficiaries of the project.

Indicators, which are established as success criteria for the outputs and objectives and help measure the impact of the project supported by the grant.
Indicators are set by the NGOs during the proposal writing (through the LOG frame process). They are reviewed by the experts during the evaluation process and finally agreed by the REC and the NGO at the winners meeting. Indicators should:

be substantial in relation with the outputs and objectives (measuring quality, quantity, time, location and target);
refer to changes achieved during the project implementation
be factual rather than subjective;
based on existing data.

Indicators can be quantitative or qualitative. Both types contribute to measuring the success of the project. Some of the quantitative indicators may include: number of species protected; hectares of land protected; increased number of visitors, participants or stakeholders; number of events, etc. Examples of qualitative indicators include involvement of stakeholders; extent of public participation; level of cooperation; increase in knowledge and awareness; improvement in services; and quality of events, quality of the publications, products or other outputs, etc.

Relevance, which gauges the degree to which priorities throughout project implementation match the broader priorities of the call. Relevance assessment involves:

checking project changes to see if they are relevant to the project purpose and goal; and
matching priorities between the project and the region/ location (and/or donor and/or REC).

Efficiency, which measures to what degree the outputs achieved derive from efficient use of financial, human and material resources.

Effectiveness, which compares expected results with the achievements and outputs made during the timeframe of the project.

Sustainability, which checks to what extent the beneficiaries can maintain the project results and whether the project has long-term viability. Sustainability assessment involves an

assessment of the the sustainability of the project results
checking to what extent the project can be replicated
check the potential for follow-up (new initiatives which were generated by the project in each of the countries involved).

Lessons learned, which examines the value of the implementation experience and the way problematic issues were dealt with. An assessment of the project lessons looks at the overall conclusions of the project.

The results of the REC assessment of grants results are indicated in the national grants report and become part of the final report to the donor.

Task	Responsibility	Days
Evaluate the grants results.	COGM/ COFA

Related policies

Supporting docs

Annex II – National grants guidelines template

Financial