close
Share with your friends

Midterm evaluation on the existing evidence behind IDH's impact stories

Midterm evaluation of IDH's impact

The purpose of IDH, the Sustainable Trade Initiative, is to improve the economic, social and environmental sustainability of global supply chains by organizing and co-funding new business models with companies and governments. IDH recently published its midterm evaluation report, a study conducted by a team from KPMG Sustainability.

Jerwin Tholen

Director Sustainability

KPMG Nederland

Contact

Gerelateerde content

The report contains the impact evaluation of IDH’s entire portfolio of programs which is aimed at improving the lives of millions of farmers and protecting landscapes based on five impact themes — smallholder inclusion, mitigation of deforestation, gender equality and empowerment, responsible agrochemical management and living wage and working conditions. The evaluation should guide IDH in the improvement of its programs and future strategies, and enhance their impact.

We evaluated the approach and collaboration with Steven Collet (IDH, Operational Director), Annemieke Burmeister (IDH, Senior Manager Monitoring and Evaluation), Jerwin Tholen (KPMG, Director Sustainability) and Brigitte Campfens (KPMG, Manager Sustainability).

According to the Annual Report, IDH works in about 40 countries, with 12 commodities and runs 12 landscape programs. How is it possible to evaluate this in a meaningful way?

Steven (IDH): That was  our challenge; we needed a portfolio evaluation, not an evaluation of individual programs. We are halfway through the deployment of our 2016–2020 strategy. So an impact evaluation is not only key to ourselves, to help us calibrate strategy but also for our donors and other impact investors to guide the direction of large investments toward the most impactful outcomes. To my knowledge, a portfolio evaluation is quite unique; therefore we are proud of this report.

How would you describe KPMG’s response?

Jerwin (KPMG): One of our first steps was the development of a systematic assessment framework. This was key to ensure consistency and robustness since we had to evaluate the five impact themes across IDH’s sector and landscape programs in parallel in only a few months’ time. In total, we read over 500 documents and reports (a mix of third-party evaluations, project documentation and external documents), conducted a survey (sent out to almost 600 stakeholders) and interviewed 17 key stakeholders. Due to our in-depth knowledge of IDH and international supply chains, we could quickly evaluate the information. To validate our conclusions, we involved a team of independent academic experts. Each of them brought in specific expertise and sharpened discussions and recommendations.

All our findings are documented in a database and referenced in the report, which means that our conclusions can be traced back to the original data. With this, we created a level of transparency around achievements in sustainable supply chains which did not yet exist, but was urgently needed.

Brigitte (KPMG): We applied best-practice project management with clear milestones and deadlines. We started with the end in mind. At a very early stage, we agreed on report structure and the visualization of our findings to ensure there were no surprises later on. We ran two rounds of workshops with the program teams of IDH; these workshops helped us conclude collaboratively that sometimes clear outcomes of IDH’s work could be identified, but in other cases sufficient data was not  yet available.

Annemieke (IDH): The workshop approach was key to the success of our collaboration; it created internal buy-in for the results. The IDH teams felt part of a process and are now driven to improve the evidence building in the future. The visualization of findings in concise dashboards was also very helpful. Despite the complex topics, these dashboards were easy to understand and created a common language. In the execution, I noticed an extra level of analysis that went above and beyond our typical project and program evaluations.

Steven: The fact that KPMG shared a clear vision early in the process of what the structure of the report would be was especially helpful. During our milestone meetings, they gave us pushback when we needed it and did not shy away from delivering the critical messages we needed to improve our data base.

Jerwin: It was a true joint effort. Just before Christmas, IDH and KPMG committed to an extremely challenging timeframe given the complexity of the evaluation. Both parties stuck to the timelines, milestones and methodology. No concessions were made but we had constructive discussions and there was willingness on both sides to go the extra mile when needed.

Midterm evaluation on the existing evidence behind IDH's impact stories

Looking at the results of the evaluation, what was the biggest eye opener?

Steven: An important insight is that we should be more specific and realistic in the definition of our end goals and not expect intermediate outcomes. For some of our themes (e.g. living wage), there was no evidence of impact because our goals were too ambitious. When our goals are too aspirational, it is sometimes not possible to measure them.

But it was also great to see that we can already measure impact in some other fields, given that we are a relatively young program and change takes time. As an example, I think of the results achieved in West Kalimantan, Indonesia where impact at field level was measured. Also, our proven contribution across all impact themes at outcome level underpins we are on the right track. Finally, the report clearly carves out our strength — convening stakeholders to accelerate change. 

How does this evaluation help IDH moving forward?

Steven: The report indicates that it is expected we might lack evidence to prove our contribution to long-term change across all five impact themes, especially at field level — the improvement of farmer livelihood. The recommendations definitely helped us to focus on our research agenda and we will include qualitative research a lot more in the near future. That way, we hope to collect sufficient evidence to prove our impact in 2021. Since the report confirms that our approach to achieve positive change is still plausible, we will stick to these strategies.

Another important area where IDH needs to put in more effort is bridging the gap between the growing volume of available (more) sustainable produce in developing countries and the actual sourcing of this produce by global companies. Commitments are made by industry, but only this is not enough. We have flagged this in our own reports as well. It is clear that more is needed to mobilize companies and get the execution of commitments going.

How is this report beneficial to the whole sector?

Steven: At IDH, one of our strategies is to disseminate lessons learnt and share best practices. I believe both the report and the approach can be of value to others active in the field of sustainable sector transformation. It brings focus and sharpens the strategy agenda in order to deliver true impact. We are therefore eager to share the report and discuss our learnings with companies, donors and other impact investors.

Profile IDH

Since 2008, IDH, the Sustainable Trade Initiative, has been contributing to improvement of the sustainability of production systems in developing countries through supply chains.

To this end, IDH deploys several strategies which are as follows:

  • Convening: IDH bundles public and private powers, finance and knowledge  to solve complex issues and unlock large-scale sustainable production and trade.
  • Co-finance: Through co-financing and de-risking arrangements , IDH triggers private investments in  sustainable market transformation.
  • Learning & Innovation: IDH pilots new business models, and evaluates and disseminates lessons learnt and best practices.

As a result:

  • 5 million farmers (35% women) were reached through training and other services;  
  • 5,6 million hectares were brought under sustainable production practices;
  • 8,5 million hectares of forest are sustainably managed;
  • 250 million euro of private sector investments were leveraged in IDH’s programs.

IDH is on track to meet most of its 2016–2020 targets to date (IDH Annual Report 2018).

Read more articles and blogs about this subject

We will keep you informed of our publications and articles by email.
Set your preferences here.

Neem contact met ons op

 

Wilt u een offerte van ons ontvangen?

 

loading image Offerteaanvraag (RFP)