A Digital Data Ecosystem to evaluate blended CPD in Rwanda | by VVOB
Together with the University of Rwanda – College of Education (UR-CE), Rwanda Basic Education Board (REB), VVOB – education for development offers CPD programmes to educators in Rwanda.
As part of these programmes, VVOB builds the capacity of REB and UR-CE to evaluate the CPD programmes. The evaluation system relied on standalone, paper-based instruments which limited the ability to gain access to real time data and to connect datasets. To improve the evaluation system, VVOB and partners developed a digital data ecosystem.
The system is made up of interconnected information technology resources that allow for a fully digital process, from data collection to reporting. The digital data ecosystem has resulted in fewer errors and faster feedback loops. The system is built on a framework developed by Kirkpatrick & Kirkpatrick (2015) which states that an evaluation of a training programme (e.g. CPD programmes) should focus on four levels:
Level 1 Reaction: this level focuses on the degree to which participants react favourably to the CPD. Examples of indicators are satisfaction, engagement and relevance.
Level 2 Learning: this level focuses on understanding whether the CPD has increased the knowledge and competences of participants. In addition, it is also important to evaluate commitment and confidence at this level, as these two indicators help to close the gap between learning and behaviour.
Level 3 Behaviour: this level focuses on the degree to which participants apply what they have learnt on the job.
Level 4 Results: this level looks at whether the CPD has had an impact on the overall (school) environment.Examples of indicators are improvements in the application of the five standards of school leadership and an improvement in school culture.
In addition to these four levels, an extra level has been added: Level 0 Participation. This level studies the level of participation in the blended CPD programme.
The digital data ecosystem is made up of components that are integrated in a dashboard that includes Key Performance Indicators for each level. The process to generate the dashboard has been digitised. Figure 19 shows the digital data ecosystem architecture, the software that has been used and how it is being applied.
