Background
Researchers in the group of Professor Wouter Buytaert (Civil and Environmental Engineering) focus on the impact of environmental change on the water cycle and its consequences for managing water resources. This work involves gathering and processing time-series data like water level, flow and temperature from various monitoring stations based in mountainous areas such as the Andes and Himalayas.
The group has previously used software donated by the Foundation for the Protection of Water (FONAG) in Quito, Ecuador to manage their data. The software uses the Django web framework and has many features for parsing and validating data, generating reports and much more. In order to make this large codebase - which had been built up over many years - more widely usable, extensible and sustainable, significant modifications would be required.
Our Contribution
The first step was for the RSE team to perform a thorough review of the existing code, identifying the critical parts that should be the focus of this first round of development and ported to the new Paricia code. We worked closely with the original developers of the FONAG software to ensure the existing user base would be able to use Paricia with minimal additional training. The code itself was translated from Spanish into English, refactored to make the code more DRY and updated to make use of modern Python libraries.
Next, we were able to add features including a user management framework allowing for different roles and permissions for logged in users, and a RESTful API so that existing frontend software could be adapted for the new system. We worked closely with the frontend developers from Kathmandu Living Labs during this part of the project.
Outcomes
Overall, this first round of development gave rise to the Paricia hydroclimatic data management system. The system is capable of efficiently parsing large files in a variety of formats and storing the time-series data in a TimescaleDB database. It is also possible for users with appropriate permissions to manage data as well as other key objects like sensors, stations, variables and other users.
By adhering to software engineering best practices including documentation, containerisation, unit testing and continuous integration, Prof. Buytaert and his team now have a solid foundation on which to build. Future rounds of development will focus on adding new features for data processing, validation and reporting.
Testimonials
Prof. Wouter Buytaert, Paricia project manager:
“In large parts of the world, the lack of good quality data is still a major bottleneck to sustainable water management. We team up with local stakeholders, including NGOs and community members, to implement participatory monitoring. The Paricia platform is an essential tool to support these activities, and to generate and manage science-quality observations.”