Background
Dr Heather Graven's research team focuses on studying the global carbon cycle and its response to human activities. Among other activities, their lab in the Department of Physics at Imperial continuously monitors the composition of the air at Imperial’s South Kensington Campus with a measuring site in the rooftop of the Huxley building. The air is pumped from there into the lab, where it is analysed by two PICARRO gas analysers. A weather station monitors other parameters like temperature, humidity, etc. Processing the data and uploading it to a safe location was done manually, so there was a need to automate this process.
Our Contribution
The RSE Team created AGDP (Advanced Greenhouse Gas Data Processing, internal access only), a small command line tool written in Python with the purpose of going through this process in an easy way. It allows to average and combine the data file from the different sources in a consolidated file, handling the time index appropriately and pull/push data to a remote repository using rclone under the hood. It also includes instructions on how to automate this process using the system scheduler (crontab in Ubuntu and the Task Scheduler in Windows). Pyinstaller was used to create the stand alone executables for both platforms.