The Research Software Engineering team at Sheffield has worked on projects involving a variety of methods and technologies, below are some of the projects we have either worked on or are currently working on:
Scientific models possess several properties that make them notoriously difficult to test, including a complex input space, long execution times, and stochastic behaviour. Furthermore, the experimental nature of many models can make it difficult to precisely specify their expected behaviour in advance. This makes them impractical to test effectively using existing techniques. The CITCoM solution is to specify tests in terms of the expected relationships between the variables within the model, for example increasing the infectiousness of a disease should lead to there being more observed cases and employ a statistical methodology known as Causal Inference to validate these relationships using pre-existing runtime data.
RSEs played a key role in the project from the outset by transforming the techniques developed by the researchers into a suite of production-ready tools to help real developers test their models. The Causal Testing Framework has automated release pipelines for both PyPI and Conda Forge, and is supported by online documentation and tutorials developed by the RSEs. This has ensured high-quality, reproducible research outputs and has led to a broader uptake of the tool by the modelling community that would not have been possible without RSE involvement.
In 2024, the energy regulator; Ofgem, mandated that all the Distribution Network Operators (DNOs) publish aggregated smart meter data publicly. GridLive is a central database and restful API for all the DNOs smart meter data that is optimized for large scale data retreival and analysis. The user can extract smart meter data based on date ranges, license area, DNO and grid reference. All the metadata from the electrical substations are also available.
The RSEs were responsible for building the full data pipeline and designed the database structure for optmised data retrieval. The software is written so the database is automatically updated every month from the DNOs data portals. The RSEs also ensured the process of building the database and API is heavily documented so the research group can edit and maintain the software.
GOTO (https://goto-observatory.org/) is a global network of robotic, wide-field survey telescopes designed to detect optical counterparts to gravitational-wave sources. Since 2023 four GOTO telescopes have been installed at two sites: GOTO-North on La Palma in the Canary Islands, and GOTO-South at Siding Spring Observatory in Australia. The GOTO Collaboration includes 10 institutions from the UK, Australia, Spain, Finland and Thailand, with ongoing operations funded by the UK Science and Technology Facilities Council (STFC).
RSE contribution to the project from Sheffield involves developing and maintaining the GOTO Telescope Control System, a custom software suite written in Python that automates the operation of the GOTO telescopes. This includes developing new scheduling algorithms and improvements to the hardware control systems, as well as ongoing optimisations and maintenance.
HYBRID is research software developed by Dr Alessandro Leonardi and his team which form part of the Geotechnical Engineering group. Written in C++ with OpenMP, the software couples the Discrete Element Method (DEM) with the Lattice Boltzmann Method (LBM) to simulate complex interactions between particles and fluids. These simulations help researchers study how granular fronts form in free-surface flows, improving understanding of particle-laden flow behaviour in environmental and geotechnical engineering.
Performance specialist Robert Chisholm assessed and optimised the existing codebase. Like many research tools created by domain experts without formal software engineering training, the original implementation contained small inefficiencies that significantly affected performance. After optimisation, the CPU version of HYBRID ran the largest tutorial model up to 6 times faster, with actual speed-ups varying by model scale and configuration.
A valuable side benefit of this work was converting HYBRID’s ParaView outputs from ASCII to binary format. Although this only slightly improved HYBIRD’s overall runtime, it allowed ParaView to load individual frames 40 times faster, dramatically streamlining the workflow for researchers analysing results from HYBRID.
The remaining project time was spent developing a GPU-accelerated prototype, as LBM simulations are particularly well suited to GPU parallelism. By the end of the project, a fully functional GPU re-implementation of both the LBM and DEM components that matched the outputs of the existing examples had been completed. Further work however remains to improve the usability and maintainability of this prototype, which the team hopes to continue with future funding.
On a 3060 RTX GPU, the GPU prototype ran the largest tutorial model 24 times faster than the original CPU code, with performance limited primarily by the use of double-precision floating-point arithmetic. On the H100 GPUs available on Stanage, the same model ran 180 times faster than the original CPU version, demonstrating the substantial potential of GPU acceleration for this research.
Collaborating with the Research Software Engineering (RSE) team has been a highly constructive experience that has made a tangible difference on how my research group works. I am a civil engineering researcher, and I often rely on numerical methods, but the challenges we face with our in-house software, particularly around performance and usability, require a level of software engineering expertise that goes well beyond what is typically available within our research group.
Alessandro Leonardi
The RSE team brought in-depth technical knowledge that helped us identify and address several key bottlenecks in the code. In particular, they supported the development of a GPU-enabled version of our fluid solver and contributed to improving the memory management routines. These changes led to a noticeable reduction in simulation times and improved the stability of our models. I was lucky to work with a Software Engineer who is an expert in this type of implementation, which made the process very smooth.
Beyond the technical improvements, the collaboration helped raise the standard of software development within our group. Working with the RSE team encouraged us to adopt more systematic development practices, improve our documentation, and consider long-term maintainability. This has directly benefited our PhD students and research staff, who now engage with the codebase more confidently and productively.
Compared to othe means of measuring volcanic SO2 emissions, ground based cameras provide a high degree of temporal and spatial resolution. However, their usage has been limited partly due to the cost of the instrumentaion. The UV PiCam system was developed by Wilkes et al. (2017) as a way of addressing this by using off the shelf components to create equipment that produces equivalent results while reducing the cost by an order of magnitude.
The RSE and DAS teams have contributed to improving both the hardware and software used as part of this project. RSE work on the software has improved its usability and robustness, and ensured that the results produced are replicable. Hardware work has entailed upgrading from the Raspberry Pi 3 to Pi 5 board, and enabling use of a different spectrometer.
Having a RSE work on a project means that we can focus and concentrate on the scientific goals knowing that we have an expert in coding (software) to deal with that side of things … we have certainly learnt more about the best practice of developing software and how to do so rigorously so there is a bit of a long lived legacy too. In short, in terms of software provision, we wouldn’t have got to this level of technology readiness as quickly.
Tom Pering
Initially developed as a prototype during Dr. Moody’s AHRC and Historic England funded PhD, PolyChron is a GUI application designed to facilitate the analysis and archiving of archaeological dating evidence.
PolyChron supports the management of both relative and absolute dating evidence, enabling users to build multiple chronological models within a Bayesian modelling framework. A key objective in developing PolyChron was to semi-automate archiving reusable archaeological dating evidence, ensuring compliance with the FAIR principles.
As part of the 2024 RSE collaboration call project, PolyChron underwent significant refactoring and restructuring, resulting in:
This collaboration has continued through the supervision of two N8CIR funded internships by Dr. Moody and Dr. Heywood.
Having access to funded RSE time was instrumental in pushing my PhD research forward following the completion of my PhD. As a teaching-only academic, I have very limited time to commit to furthering the work I started during my PhD. Also, despite being a fairly capable programmer, my codebase required extensive refactoring before I could hope to publish the software to the user community. Being able to have someone who has the expertise about best practice when it comes to writing code and publishing packages to take the refactoring project means that I saved a significant amount of time compared to if I had attempted it myself.
I was incredibly surprised by how quickly the RSE (Peter Heywood) was able to hit the ground running with the project and just how much progress they were able to make! During the project, not only was the codebase completely refactored, but Peter also made significant progress with improving the documentation, which was something that had certainly been neglected during my PhD research. I’d encourage any researcher to make use of the RSE team when funding calls become available!
Bryony Moody
For queries relating to collaborating with the RSE team on projects: rse@sheffield.ac.uk
Information and access to Bede.
Join our mailing list so as to be notified when we advertise talks and workshops by subscribing to this Google Group.
Queries regarding free research computing support/guidance should be raised via our Code clinic or directed to the University IT helpdesk.