Data collection, data use and (monitoring) systems building: the SMARTerWASH experience

Published on: 08/05/2017

Recently we closed the SMARTerWASH project (2014-2016), which was a huge joint effort of IRC, Community Water and Sanitation Agency (CWSA), Akvo, and SkyFox Limited to scale up and consolidate rural water sector’s monitoring system in Ghana. The project focused on developing ICT monitoring tools and systems and on data collection, processing and analysis at scale. While writing the end-report it was good to look back and reflect on the scale of the project and the challenges faced and ahead.

Where did the SMARTerWASH project come from?

The SMARTerWASH project did not fall from the (blue) sky. It built for example on the Triple-S Project. At the start of Triple-S, IRC and CWSA were unsure of the quality of water services being provided when measured against national standards, because this data did not exist. Therefore indicators were developed to measure and monitor water services, based on national norms and guidelines. The indicators and data collection tools were tested and refined by piloting them over a three-year period (2012-2013-2014) in three pilot districts. This resulted in CWSA publishing this indicator set as the national monitoring framework in Ghana. The indicator framework and data collection tools developed under Triple-S were a good step forward in the development of the monitoring system, but work remained to be done on its ongoing application at scale and the ICT systems required for this. This is where SMARTerWASH came in. The ICT development under SMARTerWASH built on existing ICT systems: CWSA’s DiMES (District Monitoring and Evaluation System), Akvo’s FLOW (a smartphone platform for data collection) and SkyFox’s SMS-based system for tracking functionality and ordering spare parts. 

SMARTerWASH successes: ICT development and baseline data collection at scale

The SMARTerWASH project provided the opportunity to strengthen the ICT infrastructure, by further developing the three above mentioned ICT systems and by linking them and ensuring interoperability. Furthermore, the project provided an opportunity to test (baseline) data collection at scale. By leveraging funding from other initiatives (notably World Bank’s Sustainability Rural Water Services Project, Hilton’s Strengthening Local Government Capacity Project and UNICEF’s Sanitation Knowledge Management Initiative) data collection could be applied at a larger scale than initially foreseen within the SMARTerWASH Project: in 131 (of a national total of 216) districts in 8 regions, instead of only 3 regions, as originally intended. This involved the training of national, regional and district staff, who were involved in collecting data from 23,001 handpumps, 938 piped schemes, almost 15,000 Water and Sanitation Management teams (WSMTs) and 131 service authorities in these districts. This data was processed and made available in the form of regional and district level factsheets and an online atlas.

The challenges of going beyond baseline data collection

The project showed that it does take money (the trainings, transport, per diems and phones for data collection amounted to about 5950 Euro per district, which amounts to an average for about 0.10 Euro per person served) and effort (with six district staff spending about 20 days each on baseline data collection). But as long as these are available (e.g. in the form of project funding and capacity), the ICT infrastructure can be improved and data will be collected, and, if well facilitated, even processed and analysed. The main challenge is ensuring ongoing monitoring. Obviously, ongoing monitoring requires ongoing financing in order to cover per diems, transport, the Akvo FLOW subscription etc, to the tune of about 3235 Euro per district (so about 28 Euro per Water and Sanitation Management Team per year, or about 0.05 Euro per person served). However, the challenge of ongoing monitoring is not the costs themselves, but the capacities, motivations, incentives, financial and institutional frameworks and mechanisms needed for this. Under SMARTerWASH, a start was made with addressing this. Data collection tools were successfully piloted for continuous monitoring (i.e. revisiting facilities and updating earlier collected information) and protocols for ongoing monitoring were discussed. However, ensuring that the required capacities, logistics, motivations, incentives etc. were in place at district level takes time and goes beyond the mandate of CWSA (and the rural WASH sector in general).

Another challenge is the use of monitoring data. Obviously, the intention of monitoring is not to produce endless quantities of data for the sake of it, but to produce data which is used to inform decision making in planning, corrective actions, regulation and policy making for ensuring good quality WASH service provision. There is, mostly anecdotal, evidence that the baseline data has indeed been used to inform planning and corrective actions in several districts (See SMARTerWASH stories). Monitoring data has for example been used in 11 districts in the Upper West, Upper East, Western, Brong-Ahafo and Northern Regions to inform District Water and Sanitation Plans (DWSP). It has also informed repairs and rehabilitation of over 600 boreholes with hand pumps restoring water services to an estimated 180,000 people and has stimulated several District Assemblies to form or reconstitute WSMTs (e.g. reconstitution of 203 WSMTs in Hilton districts and 24 in UNICEF districts).

This raises the question whether the use of monitoring data at district level indeed led to better services. This question is difficult to answer from the data of the SMARTerWASH Project alone, as it requires multiple data sets from the same districts. Combining handpump functionality data from Triple-S (2012-2014) and SMARTerWASH (2015-2016) from two districts where these multiple data sets are available does not show the expected improvement in functionality. Functionality in Sunyani West stays at more or less the same level, while in East Gonja functionality actually went down. Based on this, the answer to this question seems to be “no”. Just having the data is not sufficient to ensure improved services. Although it is an essential part of the puzzle, districts also need human, financial and logistical capacity, motivation and incentives to be able to use the data for improving services.   

So what have we learnt?

The SMARTerWASH project has made a great contribution to the development of the monitoring system in Ghana. However, ICT development and baseline data collection, even at scale, is the easy part. The difficulty is putting in place the “systems” required for ensuring ongoing monitoring and the use of monitoring data: the tools, capacities, logistics, but also the district, regional and national level mandates, motivations, incentives etc.

The SMARTerWASH experience clearly shows that building these systems takes time. The process of building the monitoring system in Ghana was not started under the SMARTerWASH project and has not fully completed with the closure of the project. It also shows that improving WASH sector monitoring requires systems building beyond the WASH sector.