Writing research software can be like trying to build the Suez Canal with a spoon.
Take just one mathematician (or biologist, or geophysicist, or engineer) with a large dataset.
Feed the data into the computer. Press a button and – bingo. Output award winning research.
If only it were so simple. In practice, one of the biggest challenges those of us working in mathematics, science and engineering face is the number of hours we spend keying in thousands of lines of complex, often monotonous and repetitive, source code.
The difficulty of developing fast and reliable mathematical software for science and engineering applications cannot be overestimated. Data applications require complex, high performance computing that can only be created by technology specialists – while simultaneously requiring the expertise of the application specialists for whom the software is developed. Practitioners in this area have to balance specialisation with the need to be highly multidisciplinary – and those who can do so effectively are so rare that they are often referred to as ‘unicorns’.
On the surface, the obvious solution would appear to be to train more domain specialists as software developers. However, estimates of how many lines of source code can be written per day by a professional are as low as 10 lines per day, while even moderate application codes in science and engineering can have hundreds of thousands, and even millions, of lines of source code. Indeed, I have spent nearly 20 years writing and re-engineering hundreds of thousands of lines of parallel mathematical software – an experience I can only describe as trying to build the Suez Canal with a spoon.
Which is why the work we are doing in Computational Science is so exciting. We are creating software to write software – thousands of lines of highly bespoke, optimised and verified software are written in fractions of a second. Rather than trying to create an army of unicorns and giving them spoons, we design ‘domain-specific’ computer languages that allow subject matter experts to develop specialist software using high-level abstractions native to the subject matter.
Working with Professor Paul Kelly’s team in the Department of Computing and funded by Intel, we have developed an open source domain-specific language called Devito. Successful, domain-specific languages allow complex algorithms to be expressed at a high level in very few lines of source code; bespoke compilers transform this into low-level (verbose), and highly optimised, software. This completely revolutionises not only productivity in software development, but also our relationship with computers and specialists from different disciplines.
By creating domain-specific languages that enable subject experts to express their methodology in a fashion that is native to the domain, they can essentially say: “This is my method, this is the data – create my bespoke software and compute my solution, fast.” Our mission is to enable scientists to take ideas and be able to create production-ready software within hours rather than months, and Imperial is the perfect place for this to happen.
Our mission is to enable scientists to create production-ready software in hours not months"
The collaboration between computer scientists and the domain specialist is making us completely rethink how we program computers. The ability to translate high-level algorithms or methodologies into low-level code is becoming so advanced that we are getting closer to the day when application specialists can write their equations and algorithms into research papers – and the paper itself would be directly compiled into software that can be executed. That would revolutionise the reproducibility of computational-based research and the adoption of cutting-edge innovation throughout the academic community, and technology transfer into industry.
I passionately believe that it is time to stop digging trenches with spoons and forks when there are diggers standing by ready to do the drudgery. Taking a historical perspective, automation has never been about replacing humans – it is about enhancing human capabilities so we can focus our energies on problems that require more creativity. And that collaboration between the creative human, software that writes software and machine learning will change the world.
Dr Gerard Gorman is Reader in Computational Science in the Faculty of Engineering.