How we conduct Impact Assessment

In its work, Bioversity plays a range of roles, from primary researcher to coordinator or facilitator of research activities through the preparation of guidelines and capacity building activities. 

Whatever the role, our project managers develop impact pathways that link the research with the generation of outputs (either research outputs such as journal articles, databases, guidelines or other outputs such as policy advice or public awareness), and the dissemination and uptake mechanisms of those outputs.

These pathways help the scientist to analyze and plan for the likely changes that might be generated by the intervention and that will subsequently be reflected in measurable impact.

The impact assessment methodologies used by Bioversity differ according to the nature of the expected impact and the indicators under analysis. Studies are externally peer-reviewed to ensure a high standard of work and to meet standards required by donors, the Board of Trustee and the Standing Panel on Impact Assessment of the Science Council.

In most cases, we attempt to include in each study the following:

Attribution Analysis:

In order to assess Bioversity’s role among its partners in collaborative research activities, data from different types of sources within the collaborative team and outside of it are gathered using appropriate methods (e.g. key informant  interviews, official documents, peer review publications, unofficial documents) and then cross-checked or ‘triangulated’ to assess the extent of consistency among  findings. Both attribution of outcome to the project intervention and attribution of the outcome to the implementing agency are considered.

Counterfactual Analysis:

The aspect of what would have happened without project intervention or without the involvement of the implementing agency is also addressed. A counterfactual addresses the problem of missing data associated with a project intervention when participation in the intervention cannot be randomly allocated: differences between those who participated in the project and those who did not are observable, but it is impossible to observe the status of participants in the absence of the project. Differences due to the influence of the project are easily confused with pre-existing differences between participants and non-participants. Having a proper counterfactual is essential to correct for this problem.

Baseline Analysis:

Currently Bioversity’s project teams are encouraged to collect baseline data when the project is of a large enough scale to expect significant outcomes or impacts and when donors are supportive of such activities. Support is also provided to project teams to ensure that data and sampling are of high quality so that it can provide a firm basis for future impact assessment.

Statistical design:

As participation in project interventions usually cannot be randomly allocated due to ethical, logistical or practical considerations, statistical techniques are applied to correct for selection bias in project participation and placement.

Testing new methodologies:

In some cases new methodologies are required to assess our work. We develop, adapt and test new methodologies when needed. Bioversity also hosts the Institutional Learning and Change Initiative, which is an inter-center initiative that is developing new impact assessment methodologies for collaborative research. Bioversity's own impact assessment program is benefiting from this close association.