How did conservation agriculture go to scale? A case study in utilization-focused evaluation
Article Sidebar
Main Article Content
Abstract
Background
The Foodgrains Bank has an established record working in agriculture and food security with resource constrained, marginalized farmers in sub-Saharan Africa. The three outcome areas of the Scaling-Up Conservation Agriculture in East Africa (SUCA) Program were: the adoption of conservation agriculture systems, an enabling institutional environment, and the promotion of enabling policies. These program areas were expected to yield intermediate outcomes that, together, would lead to the ultimate outcome of improved food security and sustainable livelihoods for smallholder farming households in East Africa. This case study reports on the end-line evaluation of the five-year program.
Purpose
To illustrate the overlap between utilization-focused evaluation (UFE) and collaborative approaches to evaluation (CAE). The case study profiles an agricultural intervention, and explores how the evaluation design accommodated the systemic nature of the program.
Setting
Scaling-Up Conservation Agriculture in East Africa (SUCA) was a five-year program of the Canadian Foodgrains Bank implemented from 2015-2020 to expand the size and scope of Foodgrains Bank’s work in conservation agriculture in East Africa. The program supported local partners with a target of 50,000 male and female farmers practicing a minimum of 2 of 3 conservation agriculture principles, and to improve food security and sustainable livelihoods for 18,000 of these farmers’ households across three countries.
Research design
The Foodgrains Bank was directly involved in the evaluation design through the definition of evaluation uses and key evaluation questions. Eleven implementing partners in East Africa were involved in primary data collection and some initial analysis.
Data collection and analysis
A mixed method approach was used combining quantitative, qualitative, and participatory / visual data collection tools. A robust, intersectional gender lens was applied to the data collection instruments in the form of gender disaggregated data collection and gender-focused questions across most data collection instruments.
Findings
The collaborative process confirmed a sense of ownership by the primary evaluation users over the evaluation design. The evaluation design combined outcome and learning uses that took advantages of the implementing organizations’ commitment to learning. The findings demonstrated the value of the program and produced a framework illustrating the multi-disciplinary approach underlying its success.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Arensman, B. (2020). Advocacy outcomes are not self-evident: The quest for outcome identification. American Journal of Evaluation, 41(2), 216–233
Arkesteijn, M., van Mierlo, B., Leeuwis, C., (2015). The need for reflexive evaluation approaches in development cooperation. Evaluation 21, 99–115.
Arnon, I. (1972). Crop production in the dry regions (Vol. 1–2). Leonard Hill.
Barnes, M., Matka, E., & Sullivan, H. (2003). Evidence, understanding and complexity: Evaluation in non-linear systems. Evaluation, 9(3), 265–284.
Brodhead, D. & Ramírez, R. (2014, March 20–21). DECI as a case study: Learning our way into utilization focused evaluation [paper presentation]. CDI Conference: Improving the use of M&E processes and findings, Wageningen, the Netherlands. http://www.managingforimpact.org/dal-brodhead-and-ricardo-ram%C3%ADrez-new-economy-development-group-deci-case-study-learning-our-way-util
Bryson, J. M., Patton, M. Q., & Bowman, R. A. (2011). Working with evaluation stakeholders: A rationale, step-wise approach and toolkit. Evaluation and Program Planning, 34, 1–12.
Corbeels, M., Naudin, K., Whitbread, R. K., & Letourmy, P. (2020). Limits of conservation agriculture to overcome low yields in sub-Saharan agriculture. Nature Food, 1, 447–454.
Cousins, J. B., Whitmore, E., Shulha, L., Al Hudib, H., & Gilbert, N. (2015). Principles to guide collaborative approaches to evaluation. Indicators document. Centre for Research on Educational and Community Services. University of Ottawa. https://evaluationcanada.ca/sites/default/files/20170131_caebrochure_en.pdf
Derpsch, R., Franzluebbers, A. J., Duiker, S. W., Reicosky, D. C., Koeller, K., Friedrich, T., Sturny, W. T., De Moraes Sá, J. C., & Weiss, K. (2014). Why do we need to standardize no-tillage research? Soil and Tillage Research, 137, 16–22. doi: 10.1016/j.still.2013.10.002
Douthwaite, B. & Hoffecker, E. (2017). Towards a complexity-aware theory of change for participatory research programs working within agricultural innovation systems. Agricultural Systems, 155, 88–102.
Douthwaite, B., Kuby, T., van de Fliert, E., & Schulz, S. (2003). Impact pathway evaluation: an approach to achieving and attributing impact in complex systems. Agricultural Systems, 78, 243–265.
Entz, M. H., Stainsby, A., Riekman, M., Rempel Mulaire, T., Kimathi Kirima, J., Ngotio, D., Salomons, M., Nicksy, J., Mutinda, M., & Stanley, K. (2022). Farmer participatory assessment of soil health from conservation agriculture adoption in three regions of Africa. Agronomy for Sustainable Development.
Eyben, R. (2013). Uncovering the politics of 'evidence' and 'results'. A framing paper for development practitioners. www.bigpushforward.net
Eyben, R. (2008).Powerr, mutual accountability and responsibility in the practice of international aid: A relational approach. IDS Working Paper 305.
FAO. (2015). Building resilient agricultural systems through farmer field schools. Integrated Production and Pest Management Programme (IPPM). FAO. https://www.fao.org/3/i4411e/i4411e.pdf
FAO. (2020). Conservation Agriculture. http://www.fao.org/conservation-agriculture/en/
Giller, K. E., Andersson, J. A., Corbeels, J., Kirkegaard, J., Mortensen, D., Erenstein, O. & Vanlauwe, B. (2015). Beyond conservation agriculture. Frontiers of Plant Science, 6(870). https://doi.org/ 10.3389/fpls.2015.00870
Giller, K. E., Witter, E., Corbeels, M., & Tittoneel, P. (2009). Conservation agriculture and smallholder farming in Africa: The heretics view. Field Crops Research, 114(1), 23–34.
Glover, D., Sumberg, J., & Andersson, J. A. (2016). The adoption problem; or why we still understand so little about technological change in African agriculture. Outlook Agriculture, 45, 3–6.
Glover, D., Sumberg, J., Ton, G., Andersson, J., & Badstue, L. (2019). Rethinking technological change in smallholder agriculture. Outlook Agriculture, 48, 169–180.
Hollweck, T. (2015). [Review of the book Case study research design and methods (5th ed.) by R. K. Yin.] Canadian Journal of Program Evaluation, 31(1), 108–110.
Kassam, A., Friedrich, T., & Derpsch, R. (2022). Successful experiences and lessons from conservation agriculture worldwide. Agronomy, 12(4), 769. https://doi.org
/10.3390/agronomy12040769
Kuby, T. (2003). Innovation is a social process: What does this mean for impact assessment in agricultural research? In S. Mathur & D. Pachico (Eds.), Agricultural research and poverty reduction: Some issues and evidence (pp. 58–70). CIAT.
Leakey, L. (1936). Kenya: Contrasts and problems. Methuen & Co.
Lincoln, Y. S., Thorp, L. G., & Russon, C. (2003). The storied nature of agriculture and evaluation: A conversation. Agriculture and Human Values, 20, 267–276.
Ling, T. (2012). Evaluating complex and unfolding interventions in real time. Evaluation, 18(1), 79–91.
Mayne, J. (2001). Addressing attribution through contribution analysis: Using performance measures sensibly. Canadian Journal of Program Evaluation, 16.1, 1–24.
Mhlanga, B., Ercolim, L., Pellegrino, E., Onofri, A., & Thierfelder, C. (2021). The crucial role of mulch to enhance the stability and resilience of cropping systems in southern Africa. Agronomy for Sustainable Development, 41(2), 29.
Morgan, G. (1997). Images of organization (2nd ed.). Sage.
Morton, S. (2015). Progressing research impact assessment: A 'contributions' approach. Research Evaluation, 24(4), 405–419.
Mutenje, M. J., Farnworth, C. R., Stirling, C., Thierfelder, C., Mupangwa, W., & Nyagumbu, I. (2019). A cost-benefit analysis of climate-smart agriculture options in Southern Africa: Balancing gender and technology. Ecological Economics, 163, 126–137.
Navas, J. (2018). Programa para el fortalecimiento de Consorcios Agrícolas Regionales de Investigación Agrícola, CRIA Guatemala: Resumen y aprendizaje del proceso de evaluación. [Program for Strengthening Regional Agricultural Research Consortia in Guatemala: Summary and Lessons from the Evaluation Process.]. Presentación al Primer encuentro de experiencias de evaluación participativa desarrolladas en la región de América Latina y el Caribe. PETAS, FOCELAC y DEval. Quito, 17-22 de noviembre.
Oti, S., Adong, A., nKassa, G. M., Mhalu, G., Mulebeke, R., Muhula, S., Nyakinya, J., Maina, A. M., & Njuguna, A. (2020, May 19). Implications of the Covid-19 epidemic on impact evaluation activities in East Africa. Evidence Matters series, International Initiative for Impact Evaluation (3ie). https://www.3ieimpact
.org/blogs/implications-covid-19
-pandemic-impact-evaluation-activities
-east-africa
Page, S. L. J. & Page, H. E. (1991). Western hegemony over African agriculture in Southern Rhodesia and its continuing threat to food security in independent Zimbabwe. Agriculture and Human Values, 8(4), 3–19.
Patton, M. Q. (2008). Utilization focused evaluation (4th ed.). Sage.
Patton, M. Q. & Horton, H. (2009). Utilization-focused evaluation for agricultural innovation. ILAC Brief, 22. Rome: The Institutional Learning and Change Initiative (CGIAR).
Preskill, H., Gopal, S., Mack, K., & Cook, J. (2014). Evaluating complexity: Propositions for improving practice. FSG.
Ramírez, R. & Brodhead, D.(2013). Utilization-focused evaluation: A primer for evaluators. Southbound.
Ramírez, R., Brodhead, D., Navas, J., Dhewa, C., Ramelan, V., Zaveri, S., Nyangaga, J. & Quarry, W. (2020). Comparing the validity of two sets of evaluation principles: Adding value to both. In B. Cousins (Ed.), Collaborative approaches to evaluation (pp. 185–211). Sage.
Ramírez, R., Kora, G., & Shepherd, D. (2015). Utilization focused developmental evaluation: Learning through practice. Journal of MultiDisciplinary Evaluation, 11(24), 37–53.
Ramírez, R., Quarry, W., Brodhead, D., & Zaveri, S. (2019). Evaluation & communication mentoring: A hybrid decision-making framework. Journal of Knowledge Management for Development, 14(1), 3–22. https://www.km4djournal
.org/index.php/km4dj/issue/view/34
Rogers, E. M. (1995). Diffusion of innovations (4th ed.). Free Press.
Ronner, E., Sumberg, J., Glover, D., Descheemaeker, K. K. E., Almekinders, C. J. M., Haussmann, B. I. G., Kuyper, T. W., Posthumus, H., Ebanyat, P. & Giller, K. E. (2021). Basket of options: Unpacking the concept. Outlook Agriculture, 50(2), 116–124.
Shulha, L., Whitmore, E., Cousins, J. B., Gilbert, N., & Al Hudib, H. (2016). Introducing evidence based principles to guide collaborative approaches to evaluation: Results of an empirical process. American Journal of Evaluation, 37(2), 193–217.
Sumberg, J. E. & Thompson, J. (2012). Contested agronomy: Agricultural research in a changing world. Earthscan.
Tesfaye, K., Jaleta, M., Jena, P., & Mutenje, J. (2015). Identifying potential recommendation domains for conservation agriculture in Ethiopia, Kenya and Malawi. Environmental Management, 55, 330–346.
Thierfelder, C., Baudron, F., Setimela, P., Nyagumbo, I., Mupangwa, L., Mhlanga, B., Lee, N., & Gérard, B. (2018). Complementary practices supporting conservation agriculture in southern Africa. A review. Agronomy for Sustainable Development, 38(16).
Tittonell, P., Scopel, E., Andrieu, N., Posthumus, H., Mapfumo, P., Corbeels, M., van Halsema, G. E., Lahmar, R., Lugandu, S., Rakotoarisoa, J., Mtambannengwe, F., Pound, B., Chikowo, R., Naudin, K., Tiriomphe, B., & Mkomwa, S. (2012). Agroecology-based aggradation-conservation agriculture (ABACO): Targeting innovations to combat soil degradation and food insecurity in semi-arid Africa. Field Crops Research, 132, 168–174.
Ton, G. & Glover, D. (2019). Improving knowledge, inputs and markets for legume expansion: A contribution analysis of N2Africa in Ghana and Ethiopia. IDS Practice Paper 10. Institute of Development Studies.
Ton, G., Vellema, S., Ge, L., (2014). The triviality of measuring ultimate outcomes: Acknowledging the span of direct influence. IDS Bull. 45, 37–48.
Yin, R. K. (2014). Case study research design and methods (5th ed.). Sage.