Iddo Dror, ILRI’s head of capacity development, reflects on recent workshops and their effectiveness.
Stakeholders involved in Humidtropics platforms from Africa, Asia, and Central America came together for the Humidtropics Capacity Development workshop held between April 29 and May 2, 2014, in Nairobi, Kenya. The workshops covered different approaches to agricultural innovation and were measurably successful in increasing knowledge and changing attitudes.
In this post, I’d like to reflect a bit on the learning that took place during the workshop. I should preface this by saying that the more I attend “traditional” workshops, the more I wonder whether our seemingly natural propensity for holding face-to-face workshops is the best investment of time and money in terms of learning outcomes. I believe we aren’t measuring our “ROI” on this investment rigorously enough, so it is hard to answer this question.
There’s seemingly good logic for this: a typical workshop would cost far less than the cost of running a randomized control trial on different options of delivery. However, take into account the many hundreds (possibly thousands) of such workshops that take place every year across the CGIAR system, and suddenly the picture looks (very) different.
My hypothesis is that a blended learning approach, which would involve as a minimum three core pillars – namely a pre-course e-learning component, followed by (one off or a series of) short face-to-face workshops, and complemented by ongoing mentoring over a 6-18 month period following the training would probably yield a better ROI – by delivering longer lasting impacts and demonstrated higher learning outcomes – sustained changes in knowledge, attitudes and behaviors. I’d love to have a more meaningful discussion around this on occasion. Perhaps this could be a future e-discussion topic for the CGIAR Capacity Development Community of Practice? It would be wonderful to have an evidence-based discussion about this in the not so distant future!
In the interim, when designing the Humidtropics CapDev workshop, we did try to take a quick and affordable stab at tailoring the content to the participants’ needs, and assessing the effectiveness of our training.
How did we go about it? Well, for starters, we set up a training needs assessment (TNA) via a simple survey. You can read more about how different facilitators used this info in shaping / tweaking their respective sections of the course in the full workshop report.
Next, and mainly, we started the workshop with a pre-workshop quiz, and ended the workshop with a nearly identical post-workshop quiz. This provided us with an opportunity to analyze some of the (short-term) learning that took place during the workshop. While there are certainly methodological aspects that can (and should) be looked at for such quick-fire assessments, it did provide us with some interesting aspects.
Have we cracked it? Hardly. Was this a useful exercise? Definitely (at least for me). Will this lead to more debate and exchanges on how we can improve the design and delivery of workshops at the CGIAR? Here’s hoping!
Oh, and in case you’re interested, below are some highlights for each of workshop session / days.
I’d love to know what you think – thanks in advance for letting me know!
[submit a comment at the bottom of this post!]
Day 1: Introduction to complex problems and agricultural innovation [presentation]
Agricultural problems are inherently complex. They have many dimensions, many unpre-dictable factors, and many stakeholders with potentially competing interests. Day 1 of the workshop, led by Marc Schut of WUR / IITA, aimed to stimulate new ways of thinking about agricultural problems and presented innovative strategies for tackling these prob-lems. In particular, this session introduced the Rapid Appraisal of Agricultural Innovation Systems (RAAIS) and included a RAAIS mini-workshop.
Post-workshop surveys indicate that the workshop was successful in increasing knowledge and changing attitudes. There was a shift from a technology-oriented definition of innova-tion to a systems-oriented definition of agricultural innovation. The percentage of re-spondents identifying with the latter approach increased from 33% before the workshop to 56% at the end of the workshop. Particularly notable was the participant enthusiasm for the Rapid Appraisal of Agricultural Innovation Systems (RAAIS) session; working with actu-al tools was engaging for everyone.
Participants were also asked about overall constraints in implementing agricultural inno-vation. Most commonly cited were economic, institutional, or political barriers. Biophysi-cal, technological, and socio-cultural constraints were less significant.
Day 2: Deciphering the DNA of innovation platforms [presentation]
Innovation platforms connect different stakeholders together in a network in order to achieve common goals. This workshop covered topics such as the innovation platform lifecycle, the benefits of innovation platforms, and how to monitor innovation platforms. These are new concepts for many individuals and organizations involved in Humidtropics, and many related platforms are in the initial stages. This part of the workshop focused on introducing the concepts and on making resources accessible, allowing stakeholders to approach their tasks with confidence. I led this day, with some inputs and co-facilitation from my ILRI colleague Zelalem Lema.
The pre/post quiz for this day covered topics such as the phases of innovation platforms (IP), benefits of IP, constraints in IP, and the pros and cons of existing versus new IPs. As you can see from the figure below, with the exception of the power and representation issues, all areas saw an improvement. The score on the key roles of researchers in IP, for instance, improved from 36% to 52%. On average, participants’ post-workshop quiz score was 29% better than their pre-workshop score.
This improvement, while clear in aggregate, was not shared equally amongst participants. In fact, about a quarter of participants scored (marginally) lower in the post-workshop quiz! Luckily, the vast majority did score better in the post-quiz, some scoring several times their initial score.
Day 3: Knowledge, learning and making meaning in innovation platforms [presentation]
Led by Julia Ekong from ICRA, this session consolidated the information of the first two days and explored innovation platforms further. Four frameworks for facilitators were introduced as illustrations and to capture platform dynamics.
This session faced some challenges because most of the participants were at the initial stages of establishing or implementing platforms, though some participants had significant experience in the area. The heterogeneous composition of the workshop with regards to participants’ levels of expertise meant it was difficult to evaluate what knowledge was being created (beyond technological solutions). This session could not leverage existing knowledge as much as was anticipated. In the future, this type of material would perhaps be more successful if presented to more homogeneous groups. Some more thought about quantitative metrics for assessing the learning taking place would probably also be useful.
Day 4: Reflexive Monitoring in Action (RMA) [presentation]
This session, led by Marlèn Arkesteijn (Capturing Development) discussed the three dimensions of complex problems—certainty, agreement, and systemic stability—and how reflexive monitoring in action (RMA) can support the learning processes when dealing with complex problems and innovation. The key principles of RMA were introduced, and various aspects of reflexivity were highlighted, such as change of perspective, the re-construction of reality, emergent outcomes, and turning the camera. Working with the timeline was also introduced, and the timeline was used to evaluate the workshop.
The results of the pre/post quiz indicated significantly more RMA confidence after the workshop; whereas in the pre-workshop quiz, ten participants indicated they knew only one RMA principle, that number had fallen to four in the post-workshop evaluation. Post-workshop, fifteen participants indicated on a questionnaire that RMA “really appeals to me;” only one participant felt that RMA “is not what I’m interested in.” Sixteen out of seventeen participants said, “for sure I will use some of the principles and tools.”