Text classification is a foundational problem in natural language processing (NLP), allowing us to automatically assign predefined labels to massive volumes of unstructured text. In the context of technological innovations, one of the most valuable applications of text classification is automating the assignment of International Patent Classification (IPC) hierarchies to technical documents.

Historically, this process has been a computationally expensive and time-consuming manual task. While traditional machine learning methods (like Support Vector Machines or simple Neural Networks) have been used in the past, they often require extensive feature engineering and struggle to grasp the nuanced, technical context of patent data.

To tackle this, we developed a robust, state-of-the-art methodology capable of semantically understanding technical patent abstracts and predicting their corresponding IPC labels across multiple hierarchical levels. This cross-institutional collaborative effort was led by Dr. Enrico Vanino from the University of Sheffield’s School of Economics and Prof. Carlo Corradini from the Henley Business School at University of Reading, with developer support provided by Dr. Farhad Allian from the Data Analytics Service at University of Sheffield’s IT Services, who developed the model architecture, codebase, and technical report. This post provides a look under the hood at our data processing pipeline, model architecture, and evaluation strategy used as part of this research.

The Data Challenge: Navigating Label Scarcity and Imbalance

Figure 1: Corpus statistics of the PATSTAT dataset. (a) shows the label count distribution for the Section level. The dashed line shows the mean value. 50% of labels are above the mean value. (b) same as (a), but for the Class level. Only 27% of labels are above the mean value. (c) shows the word-count distribution of the abstracts.

Training any multi-label deep learning model requires a massive amount of high-quality data, especially given the hundreds of possible hierarchical IPC classifications. To build out our classification architecture, we leveraged PATSTAT, a global database containing over 130 million patent documents. By sampling 177,000 relevant patents for this research, we curated a comprehensive training dataset mapping technical abstracts to verified multi-label IPC codes (at the broader “Section” and “Class” levels). This allowed our model to learn the complex linguistic patterns associated with diverse technological domains. Figure 1 provides an overview of the corpus statistics used in our sampled training dataset, including the label distributions for the two IPC labels used in our study, and the mean word count per abstract.

As with all text classification problems, one major hurdle in patent data is the severe class imbalance. The label space is highly skewed toward dominant fields like electronics or chemistry, while niche or emerging domains remain sparsely represented. One approach is to process the data by upsampling the frequency of low-occurring labels before training the model. However, to ensure the model accurately reflects this real-world distribution, we intentionally avoided any artificially upsampling of rare labels in our approach. Instead, we relied on strict stratified sampling across our training, validation, and test splits that mimics the raw dataset’s proportions and reduces the model’s bias. This guarantees that rare technological domains are preserved and adequately represented across all phases of model development. A summary of the data splits is shown in Table 1.

Table 1: Train-val-test statistics following the stratified sampling. Note that there exists more than one label per patent on average.

Our Approach: Fine-tuning and Hierarchical Masking

Figure 2: Schematic diagram of our workflow demonstrating the preprocessing, fine-tuning, and classification stages.

To capture the highly specialised jargon of technical abstracts, we moved beyond standard language models (like the original BERT, which is trained on general text like Wikipedia) and utilised SciBERT. SciBERT is a domain-specific transformer pre-trained on 1.14 million scientific papers, making it suitable to ingest complex, domain-specific abstracts. The architecture of our multi-label text classification pipeline involves the following three main steps:

1. Preprocessing: Raw abstracts are tokenised (up to a maximum sequence length of 512 tokens). Each token is mapped to a 768-dimensional embedding vector, and the final hidden state of the classification token ([CLS]) is pooled to create a single, dense semantic representation of the entire abstract.

2. Fine-tuning: The token embeddings are passed into the pre-trained SciBERT layer to perform fine-tuning. [CLS] token pooling is performed to represent the global information of each abstract. The pooled output passes through a dropout layer (to reduce overfitting) and feeds into two distinct, fully connected classification heads; one for predicting the IPC “Section” (the parent level) and one for the IPC “Class” (the child level).

3. Predictions: Because IPC codes are strictly hierarchical, we leveraged this property and engineered the model to mask invalid parent-child combinations during predictions. For instance, if the model confidently predicts that a text belongs to Section “A”, the subsequent class predictions are constrained to only output classes that begin with “A” ( e.g., A01). This masking approach drastically reduces the effective label space (from 124 possible classes down to just the relevant ones for that section), cutting through the noise and improving the signal-to-noise ratio. The end-to-end workflow is demonstrated in Figure 2.

The model was fine-tuned to minimise a weighted combination of binary cross-entropy losses across the two hierarchical levels with respect to the frequency of IPC labels. To handle the computational load of the 110-million parameter model, training was executed using mixed-precision (FP16) on a single node equipped with dual AMD EPYC CPUs and an NVIDIA A100 GPU on Sheffield University’s HPC cluster, Stanage.

Evaluation Strategy: Using Standard and Hierarchical Metrics

Figure 3: Training and evaluation performances of the hierarchical classifier across 20 epochs. (a) Training loss exhibits a smooth convergence. (b) Section-level F1-micro rapidly plateaus at 82% while class-level performance continues improving from 33% before stablising around 58%, highlighting the difficulty in predicting within a large label space. (c) Hierarchical F1 scores show consistent improvement with an optimisation approach providing a 3-4% gain over specified thresholds, reaching a final optimised H-F1 of 40%.

Evaluating a model trained on heavily imbalanced, hierarchical data requires looking beyond basic accuracy thresholds. During validation, we treated the task as a multi-label classification problem, looking at both micro metrics (e.g. F1-micro, which favour frequent classes) and macro metrics (e.g. F1-macro, which treat rare niche classes equally) to objectively quantify the model’s performance.

Importantly, we also implemented hierarchical metrics to expand our evaluation strategy. Standard flat metrics, such as those described above, treat all misclassifications as equally incorrect. Hierarchical metrics, however, award partial credit when predictions are taxonomically related. For instance, if the model predicts class “B01” instead of the true label “B23”, it receives partial credit because it correctly identified the broader technological domain (Section B). This proves highly valuable for mapping interdisciplinary innovations that might span multiple related fields.

Table 2: Sample evaluation metrics showing the micro (frequent labels) and macro (rare labels) scores.

Summary and Future Work

This work represents a significant step toward automating and improving the accuracy of patent categorisation at scale. It has the potential to enhance patent office workflows and enable more sophisticated analyses of technological landscapes. While the model currently serves as a research prototype, we envision deployment scenarios where it could assist patent examiners in preliminary classification tasks or support innovation analysts in mapping emerging technological trends across large patent portfolios. The framework’s modular design also makes it adaptable to other hierarchical text classification domains beyond patents. A publication of this work is currently in press.

While the model architecture establishes a powerful methodology for hierarchical text classification of patent data, there’s still room to grow. Future iterations of this approach can investigate advanced attention mechanisms to better exploit the hierarchical structure of IPC labels, incorporate richer technical details found in full patent claims rather than just abstracts, and explore specialised neural architectures explicitly designed to mitigate the extreme class imbalance inherent in patent classification. Hardware considerations may also play an important part in improving our approach. Training on our 177,000-document corpus required a relatively standard amount of computational resources ( e.g. Dual AMD CPUs and a NVIDIA A100 GPU), however, expanding to the full PATSTAT database of 130+ million patents would demand distributed training across multiple GPUs or even TPU clusters. Memory bottlenecks from the 768-dimensional embeddings and 110-million parameters could also be addressed through model distillation, quantisation, or even more efficient transformer variants.

Acknowledgements

This research was supported by the UK’s Economic and Social Research Council (ESRC) with the project name “Exploring the Link between Publicly Funded R&D Collaborations and Regional Technological Development” under the grant FFCoE005. A paper is currently in press.

Contact Us

This research was undertaken by the Data Analytics Service (DAS) team at the University of Sheffield’s IT Services. We offer free consultations, training, and technical collaboration to researchers across all Faculties.

Whether you need help with text classification, natural language processing, complex data pipelines, or machine learning, our team of Research Technical Professionals (RTPs) in the Data Analytics Service (DAS) and the Research Software Engineering Group can help.

Why use a GitHub Organisation?

If you’re already using Git and GitHub for version control (and if you’re not, there are good reasons to start), you’ve probably been working from your personal account. That works fine for solo projects, but research groups face a specific problem: what happens to the code when people leave?

When a PhD student or postdoc moves on, their personal repositories often become “gradware”—abandoned code that the group can no longer access or maintain. An Organisation solves this by giving the research group itself ownership of the code, separate from any individual’s account.

Organisations also let you:

• Manage access centrally (who can see what, who can change what)
• Set consistent security policies across all your projects
• Keep your group’s *work together in one discoverable place

The University’s GitHub Enterprise Early Access Program provides access to these features. This provides several advantages over a standard free account, including:

• Unlimited Private Repositories: Collaborate securely within your team without public exposure.
• Enhanced Security: Access to advanced tools like secret scanning to prevent accidental leaks of API keys or credentials.

To get started, you will need to follow the University’s formal onboarding process. Please refer to the GitHub Enterprise Policy Document; specifically, you will find the required setup form in Section 5 (Requesting an Organisation).

💡 Tip: When naming your Organisation, choose something descriptive and evergreen such as UoS-[Institute] or UoS-[Lab-Name], rather than something tied to a specific year or project. An organisation can contain multiple repositories that span multiple projects.

Who looks after the Organisation?

GitHub Organisations need designated Owners—people who take responsibility for the space. This isn’t just a technical role; it comes with ongoing obligations.

You need at least two owners for continuity. These people will be responsible for:

• User management: inviting new members, removing people who leave
• Security oversight: ensuring the organisation follows University policies, responding to security alerts
• Housekeeping: periodically reviewing inactive members, archiving old repositories

Choose your owners carefully. They should be people who’ll be around for a while and who understand (or are willing to learn) the security implications of managing shared code. The full list of owner responsibilities is in Section 1.3 of the GitHub Enterprise policy.

To prevent an Organisation from becoming “orphaned” when a researcher moves on, we recommend that the Principal Investigator (PI) or a permanent staff member always holds one of the Owner roles. This ensures the department retains access to the research outputs even after students or post-docs finish their contracts.

Essential security settings

Once your Organisation is created, you’ll need to configure some settings. You’ll find these under Settings in your organisation’s GitHub page. The GitHub documentation on security settings covers everything in detail, but here are the key ones to address first.

If you’re working with commercially sensitive code, personal data, or anything requiring particular protection, contact the Research & Innovation IT team to discuss your specific needs.

Restrict public repository creation

Where: Settings → Member privileges → Repository creation

What to do: Untick “Public” (leave Private and/or Internal enabled)

Why: This prevents anyone from accidentally making a repository public. Research code often needs to stay private until publication—or permanently. With this setting, making something public requires deliberate action by an owner.

Control outside collaborators

Where: Settings → Member privileges → Outside collaborators

What to do: Untick “Allow repository administrators to invite outside collaborators”

Why: Outside collaborators are people who aren’t organisation members but can access specific repositories. Without this control, any repository admin could invite external people without oversight. Centralising this with owners means you always know who has access and can ensure external users meet any contractual or ethical requirements.

Enable automated security scanning

Where: Settings → Code security and analysis

What to do: Enable “Dependabot alerts” and “Secret scanning”

Why: These tools watch for common security problems:

• Dependabot alerts you when the libraries your code depends on (Python packages, R packages, etc.) have known security vulnerabilities. It can even suggest fixes automatically.
• Secret scanning catches accidentally committed passwords, API keys, or other credentials. This happens more often than you’d think, and catching it early prevents data breaches.

Restrict GitHub Actions

Where: Settings → Actions → General → Policies

What to do: Select “Allow enterprise actions and reusable workflows”

Why: GitHub Actions runs automated tasks on your code (tests, checks, deployments). There’s a marketplace of community-built Actions, but not all are trustworthy. Restricting to enterprise-approved actions reduces the risk of malicious code running in your workflows.

Limit Personal Access Tokens

Where: Settings → Personal access tokens → Settings

What to do: Under “Tokens (classic)”, select “Restrict access via personal access tokens (classic)”

Why: Personal Access Tokens let scripts access GitHub on behalf of a user. Classic tokens have broad permissions and are a security risk if leaked. This setting forces users to approve which organisations their tokens can access and encourages use of the newer, more secure fine-grained tokens.

Require two-factor authentication

Where: Settings → Authentication security

What to do: Tick “Require two-factor authentication for everyone in your organization”

Why: Your organisation’s security is only as strong as its weakest password. Two-factor authentication means that even if someone’s password is stolen, attackers still can’t access your repositories.

Protecting important repositories

Beyond organisation-wide settings, your most important repositories deserve additional protection.

Branch protection rules

Where: Repository Settings → Branches → Add branch protection rule

What to do: Target your main branch and enable “Require a pull request before merging” and “Require approvals”. While you’re there, also turn on:

• Dismiss stale pull request approvals when new commits are pushed
• Require approval of the most recent reviewable push

Why: Without branch protection, anyone on the team—including you—can push changes directly to your main branch. Requiring a pull request means every change has to be reviewed before it lands: the code equivalent of asking a colleague to read over your work before you send it.

Common mistakes to avoid

Committing data to repositories. GitHub is for code, not data storage. Large files make repositories sluggish, and committing research data may violate ethics approvals. Use a .gitignore file to exclude data files and folders.

⚠️ Warning: GitHub is not a data repository. Never commit raw research data, sensitive participant information, or large binary datasets to GitHub. Use it for code, scripts, and documentation only.

Hard-coding credentials. It’s tempting to put database passwords or API keys directly in scripts. Don’t. Use environment variables or a .env file (added to your .gitignore) to keep credentials separate from code.

Going further

This guide covers the essentials, but GitHub offers more for making research software FAIR (Findable, Accessible, Interoperable, Reusable):

• Citation files: Add academic citations using the Citation File Format so others can credit your work properly
• Archiving with ORDA: Archive snapshots of your code in the University’s research data repository with a permanent DOI
• Automated testing: Use GitHub Actions to run tests automatically when code changes

Why use ORDA instead of just leaving code on GitHub? While GitHub is excellent for active development, it does not guarantee long-term preservation. To meet most funder requirements, you should archive “frozen” versions of your code in ORDA.

Unlike GitHub, ORDA provides a persistent Digital Object Identifier (DOI). This ensures that when you cite your code in a paper, the link will never break, and other researchers can access the exact version of the software used to produce your results.

For more information on integrating GitHub with ORDA (which is based on the FigShare platform,) please read How to connect Figshare with your GitHub account on the FigShare documentation.

Summary checklist

If you are starting a new research organisation today, follow these six steps:

1. Request Access: Complete the form in Section 5 of the Enterprise Policy.
2. Assign Owners: Ensure at least two people (including a permanent staff member) have Owner status.
3. Restrict Creation: Set “Repository Creation” to Owners Only to prevent sprawl.
4. Enforce 2FA: Check that “Require two-factor authentication” is active for all members.
5. Protect Branches: Enable “Branch Protection” for your main or master branches.

Getting help

Contact the Research Software Engineering or Research & Innovation IT team if you need help choosing the right settings, working with sensitive data, or troubleshooting permissions issues.

While the security configurations outlined above provide a robust baseline for most projects, it is important to remember that security is not a one-size-fits-all requirement. Not every option needs to be strictly locked down in all circumstances; rather, managing an organization involves making informed judgments and balancing security trade-offs against the specific collaborative needs of your research. If you are uncertain about which settings are appropriate for your project’s unique requirements, or if you need help navigating these decisions, you can contact us for further guidance

Training courses

We offer a range of short courses throughout the year that are free for all researchers. Learn beginner to advanced skills in popular programming languages, computing, and statistical packages.

As part of the FAIR² for research software curriculum, the Research Software Engineering team provides a two-tiered version control training program tailored specifically for researchers. The journey begins with the “Git, GitHub and GitKraken - From Zero to Hero” module, which introduces absolute beginners to version control basics and collaborative workflows using graphical tools. For those looking to level up, the “Git With It” session dives deeper into branch management, commit history, and advanced collaboration techniques. Together, these modules empower researchers to move beyond basic file saving, ensuring their code is reproducible, organized, and ready for collaborative open research.

Online guidance

These resources provided by other Universities may be useful as you learn to collaborate on research software using GitHub:

• How to set up GitHub for a research project by University College London
• Introduction to Using GitHub by Rachael Ainsworth from the University of Manchester

Here we review the work undertaken on the projects we supported last year and introduce the successful applications in the 2025 round.

2025 Call for Proposals

This year saw an increase in the number of applications with 35 received by the deadline, an increase from the 26 applications received in 2024. A review panel of ten RSEs from the RSE team and adjacent Data Analytics Service was convened to assess the applications and decide which should receive support. The panel enjoyed reviewing the diverse applications and had a challenging decisions to make as to which projects to support as the quality of applications was again high.

All work undertaken will be done with a view to adhering to the FAIR for Research Software Principles

Optimising & Efficiency Improvement of a National Building Material Stock

Professor Danielle Densley Tingley’s research “explores sustainable building design solutions, focusing on material impacts, so we can create a built environment that operates within the carrying capacity of the planet.”. To achieve this they are developing models of existing building stock using a hierarchical building material intensity (HMI) framework to assess what buildings are made of in the UK. The software is implemented in Python and has been applied to building stock in suburbs of London but the group is looking to extend its application to the rest of the UK. The proposed work will profile the existing code base with a view to optimising its performance, review the existing database structures with a view to improving efficiencies and implement parallelisation so that analyses can scale and leverage High Performance Computing (HPC) facilities that are available to researchers.

Yorkshire and Humber Office for Data Analytics (YOHDA)

Professor Vania Sena from the University’s Management School submitted this proposal to improve the code base that underpins YOHDA a data resources that “supports data-drive decision making through open-access data and analytics” that pulls open data from resources such as the Office for National Statistics (ONS), Department for Work and Pensions (DWP) and many others. Currently the aggregation of data is a manual process that is limiting the scalability of the project. RSE support will develop a structured SQL database server that uses Advanced Programming Interfaces (APIs) to automate the data ingress and optimise the existing transformation and cleaning routines to improve performance and stability, streamlining the workflow and reducing the overhead of maintaining this valuable resource. At the same time documentation will be developed that facilitates on-boarding and aids in up-skilling of team members.

Software for Modelling Neutrino Emission from Spent Nuclear Fuel

Spent Nuclear Fuel is stored in above-ground dry casks at interim storage facilities, where it is monitored for decades for safety and nonproliferation. Effective monitoring technologies are of great importance, as interim facilities store hundreds of tonnes of highly radioactive spent fuel, and increasing reliance on civil nuclear power has led to more interest in developing improved methods of monitoring SNF. Dr Elisabeth Kneale has developed Python software package predict antineutrino emission spectrum from Spent Nuclear Fuel but has sought the assistance of the RSE team to improve the open-source nature of the software, packaging, develop a Graphical User Interface (GUI) whilst improving compliance with the FAIR4RS standards.

Conclusion

The call for proposals in 2025 has again shown there is a diverse need for RSE support across all departments of the University of Sheffield and that increasingly researchers are recognising the benefits that dedicated RSE support can bring to the software components of academic research. The RSE team are pleased to be able to support researchers the successful applicants to this years call.

If you are considering putting in a funding application that involves writing/developing/maintaining software having dedicated RSE support can strengthen your proposal. If you would like to speak to the RSE team about support please don’t hesitate to get in touch by emailing rse@sheffield.ac.uk.

From left to right: Yuliang Weng, Martin Dyer, Daniel Brady, Michael Foster, Romain Thomas, Peter Heywood, Shaun Donnelly, Erika Siregar (PhD from the information School), Christopher Wild, Gemma Ives, Tamora James, Edwin Brown, Neil Shephard, Farhad Allian, Robert Chisholm,Matthew Leach, Joe Heffer, Twin Karmakharm, Paul Richmond

The University of Sheffield RSEs played a major role in this year’s event — not only as a Silver Sponsor, but also through leadership positions, presentations, posters, workshops, and volunteering. Our team’s involvement demonstrates Sheffield’s strong commitment to the RSE community.

Several Sheffield RSEs took on key leadership roles in shaping RSECon25:

• Twin Karmakharm – Steering Committee Chair
• Romain Thomas – Programme Co-chair
• Robert Chisholm – Logistics Co-chair
• Neil Shephard – Publicity Chair

These roles highlight the influence and responsibility our team have in driving the direction and success of the national RSE conference.

In addition, Sheffield’s RSE team was present across the programme, with contributions spanning talks, posters, walkthroughs, and workshops:

• Talks & Walkthrough:
  • Matthew Leach – AMD GPUs for Scientific Computing
  • Shaun Donnelly & Edwin Brown – How to talk to your documents: An introduction to using natural language to query documents with Retrieval Augmented Generation (RAG)
  • Romain Thomas & Neil Shephard – Running a local RSE call for proposals
• Workshops:
  • Robert Chisholm & Peter Heywood – Reasonable Performance Computing SIG (SIG-RPC): Help Identify and Document Performance Traps in Research Software
• Posters:
  • Tamora James & Romain Thomas - FAIR2 for research software: developing a FAIR (Findable, Accessible, Interoperable and Reusable) and reproducible research software training programme
  • Edwin Brown - AutoEmulate:  Python library for automatically creating accurate and efficient emulators of complex simulations
  • Dan Brady - Leveraging GitHub API Data to Evaluate Git and GitHub Training Outcomes
  • Martin Dyer - Developing the GOTO Telescope Control System

This broad and diverse participation reflects the strength of our team across technical expertise, training, and community support.

Sheffield colleagues also supported the smooth running of the conference through volunteering (Daniel Brady & Michael Foster) and session chairing (Paul Richmond & Joe Heffer from the DAS team).

Looking towards RSECon26

We are excited to announce that the University of Sheffield will host RSECon26 in 2026 at the wave (9-11th September), which will also be co-located with the first International Research Software Conference (IRSC) (7-8th September). Building on our contributions at Warwick, we look forward to welcoming the RSE community to Sheffield next year for another vibrant and impactful conference. Romain Thomas and Twin Karmakharm will be leading the conference as programme chairs.

• Credit Photo: RSE Society

A total of 26 applications were received from across the faculties of the University and the review panel which consisted of nine RSEs had robust discussions about which to fund as the quality and proposed work was of a high standard.

In the end the RSE team was able to fund support for three proposals and team members Dr Robert Chisholm and Neil Shephard are about to start work on two of the projects with the third postponed, with agreement, until a new RSE who has been recruited starts later in the year.

All work undertaken will be done with a view to adhering to the FAIR for Research Software Principles

SLAMSeq

The SubLab, which is lead by Dr Ian Subery, have developed software to analyse the output of SLAMSeq experiments which performs sequence alignment of variably spliced RNA sequences to assess degradation rates. Despite wide adoption of the technique the current software is limited to only being run on a per-gene basis rather than per isoform basis and statistical comparison of differences in decay rates are limited. The Sudlab have addressed this deficiency by developing an analysis pipeline with pre-processing performed in Python and statistical analysis undertaken in R.

RSE support will be used to refactor the code into formal packages with a modular code structure and formal tests which facilitate long-term maintenance and make extension easier and will make it straight-forward for users to install. Translation of code into a single language will be considered as this would lower the barrier to uptake by external users and documentation will be developed and deployed covering the API and the practical side of using the software.

HYBIRD

HYBIRD is software developed by Dr Alessandro Leonardi and his research group with the Geo-Technical Engineering group and is written using C++. It combines the Discrete Element Method (DEM) and the Lattice Boltzmann Method (LBM) to simulate complex particle-fluid interactions. These fluid-structure interactions have allowed the investigation of the formation of granular fronts in free-surface flows, enhancing understanding of particle-laden flow dynamics which have been successfully applied to key areas in environmental and geo-technical engineering.

Work will be undertaken by Dr Robert Chisholm, a performance optimisation and GPU parallelisation specialist from our team, to modernise HYBIRD’s build system, address the limiting factors of its performance and enable it to take advantage of GPU parallelisation. The broad objectives of this project are to reduce barriers to entry for new users and to increase the performance to enable faster and larger research experiments.

Polychron

Polychron is prototype software developed by Dr Bryony Moody of the School of Mathematical and Physical Sciences that facilitates the analysis and archiving of archaeological dating evidence. Written in Python the prototype consists of a statistical algorithm (backend) and a GUI frontend to facilitate usage.

Work will be undertaken by a new RSE who will be starting later in the year. The exact remit of the work to be undertaken has not yet been finalised but the code base will benefit from restructuring into a Python package with refactoring of the GUI to make it easier to maintain and extend and improved error detection.

Summary

As the number of applications demonstrated, the demand for RSE support across all faculties within the University is high and it was interesting to find out about the broad range of software projects being undertaken by different research groups. We wish we could have supported more of these teams.

If you are considering putting in a funding application that involves writing/developing/maintaining software having dedicated RSE support can strengthen your proposal. If you would like to speak to the RSE team about support please don’t hesitate to get in touch by emailing rse@sheffield.ac.uk.

Photo by Neil Shephard.

If you use Python regularly chances are you are familiar with and use virtual environments to isolate the installed packages as you develop your code from the Python packages installed at the system level. The standard library venv, and other packages such as pipx, virtualenv and Anaconda are all popular options. Many of the popular package development suites such as Hatch, PDM and Poetry will also manage virtual environments for you too. One you might not have come across is virtualenvwrapper which is…

a set of extensions to Ian Bicking’s virtualenv tool. The extensions include wrappers for creating and deleting virtual environments and otherwise managing your development workflow, making it easier to work on more than one project at a time without introducing conflicts in their dependencies.

NB If you are using Microsoft Windows you can follow along by using Windows Subsystem for Linux, otherwise virtualenvwrapper installation and usage is slightly different under Windows and you should consult the virtualenvwrapper.

What are Virtual Environments?

Before getting started a quick recap for readers who aren’t familiar with virtual environments.

When you run programmes, such as invoking python at the command line, whether that is GNU/Linux, Mac OSX or Microsoft Windows, the operating system searches for the command you run in a defined set of places. It doesn’t look everywhere, rather it looks in the locations defined by an environment variable called $PATH. This variable holds a set of paths to locations in which to look for executable/binary programmes. You can inspect its value with echo $PATH. The order in which the listed paths are checked takes left-to-right precedence and on my system this looks like the following.

❱ echo $PATH
/home/neil/.local/share/pnpm:/home/neil/bin:/home/neil/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/lib/llvm/18/bin:/usr/lib/llvm/17/bin:/home/neil/.local/bin:/home/neil/.cargo/bin:/home/neil/.node/bin

Thus if I invoke python the first path that is checked for a binary with that name is /home/neil/.local/share/pnpm. If one isn’t found the second is checked (/home/neil/bin), then the third (/home/neil/.cargo/bin) and so on until a match is found. If no match is found you can check the path to a given binary using which and if no programme with that name is found you are told. This means its either not installed o your system or you perhaps made a tpyo (Hint use tab-complete its usually installed and works in most shells).

❱ which python
/usr/bin/python
❱ which ptyhon
ptyhon not found

A virtual environment modifies the $PATH variable, placing a new path at the start that points to a directory under your user account that contains a copy of python and the specific set of packages you have installed under that environment. We’ll cover this in more detail but if I look at the $PATH variable after having activated the topostats virtual environment you will see that its value has been modified and the first directory that is checked is /home/neil/.virtualenvs/topostats/bin and if I check which version of python is used it is the one within this directory.

❱ echo $PATH
/home/neil/.virtualenvs/topostats/bin:/home/neil/.local/share/pnpm:/home/neil/miniconda3/condabin:/home/neil/bin:/home/neil/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/lib/llvm/18/bin:/usr/lib/llvm/17/bin:/home/neil/.local/bin:/home/neil/.cargo/bin:/home/neil/.node/bin
❱ which python
/home/neil/.virtualenvs/topostats/bin/python

Managing Virtual Environments with virtualenvwrapper

As the name suggests virtualenvwrapper is a wrapper for the virtualenv package that adds a bunch of convenience functions and features that, in this author’s view, make it easier to install, activate/deactivate and manage your virtual environments.

Installation

I recommend installing virtualenvwrapper at the system level, ideally with your systems package manager. Under different GNU/Linux or OSX using Homebrew this can be done with…

❱ sudo pacman -Syu virtualenvwrapper       # Arch
❱ sudo emerge -av virtualenvwrapper        # Gentoo
❱ sudo apt-get install virtualenvwrapper   # Debian/Ubuntu
❱ brew install virtualenvwrapper           # OSX Homebrew

…or if you are happy installing packages using pip you can follow the official installation instructions and use pip to install at the system level.

❱ sudo pip install virtualenvwrapper

Alternatively you can install locally using pip by using the --user flag.

❱ pip install --user virtualenvwrapper

You now need to ensure that the virtualenvwrapper.sh is sourced each time you start a new shell. To do this you need to add lines similar to the following to your ~/.bashrc or ~/.zshrc files depending on which shell you use (the former is the default on most GNU/Linux systems and Windows Subsystem for Linux, the later is the default under OSX).

# Configure and setup virtualenvwrapper
export WORKON_HOME=${HOME}/.virtualenvs
export PROJECT_HOME=${HOME}/work/
source /usr/local/bin/virtualenvwrapper.sh

The values of PROJECT_HOME should be specific to where you wish to create your projects, whilst the argument to source will depend on how you have installed virtualenvwrapper, you can check using which so use the value returned by

❱ which virtualenvwrapper.sh
/usr/bin/virtualenvwrapper.sh

Always using a virtual environment

Once in virtualenvwrapper is installed you can change a configuration option to pip to ensure that you are always using a virtual environment when you attempt to install packages using pip by adding the following to the ~/.config/pip/pip.conf file on your system.

[global]
require-virtualenv = True

Creating a Virtual Environment with virtualenvwrapper

This is straight-forward.

❱ mkvirtualenv test_env
(test_env) ❱

By default on creating a new virtual environment it is activated and your prompt should change to reflect this with the environment name appearing as part of the prompt. You can check that the python that will be executed resides in this directory using which.

(test_env) ❱ which python
/home/neil/.virtualenvs/test_env/bin/python

Activating/Deactivating Virtual Environment

You have been dropped into the test_env virtual environment. To exit from it use the deactivate command, again you can check which version of python is found on your $PATH using which.

(test_env) ❱ deactivate
(test_env) ❱ which python
/usr/bin/python

You won’t always create a virtual environment when you want to use it though, that would be a waste of time and bandwidth downloading and installing packages (although these are cached and used if no new updates are available). To activate an existing virtual environment you use the workon command.

 ❱ workon test_env
(test_env) ❱

Project Directories

Typically code for a project resides in its own directory and this can be automatically bound to the virtual environment using the mkproject command instead of mkvirtualenv. The project directory is stored in the $PROJECT_HOME path you will have configured during installation. You can then create a project and a virtual environment with…

❱ mkproject new_project
(new_project) ❱

One advantage this confers is that when you activate the project, e.g. with workon new_project your working directory will automatically change to that of the project, i.e. $PROJECT_HOME/new_project, meaning you don’t have to use cd to change directory to that location.

Manually setting Project Directories

If you don’t use mkproject to create a project you can set it manually after activating a virtual environment using setvirtualenvproject in the desired directory. This adds an entry to the ~/.virtualenv/<env_name>/.project file that reflects the directory associated with the environment.

❱ mkvirtualenv new_project
(new_project) ❱ cd ~/work/some/other/path/for/new_project
(new_project) ❱ setvirtualenvproject

Disabling Switching to Project Directories

The environment variable VIRTUALENVWRAPPER_WORKON_CD “controls whether the working directory is changed during the post activate phase. The default is 1, to enable changing directories. Set the value to 0 to disable this behaviour for all invocations of workon”. If this is something you don’t want to happen you should set this in your ~/.bashrc or ./zshrc, otherwise the default can be left alone.

export VIRTUALENVWRAPPER_WORKON_CD=0

Removing Virtual Environments

You can remove a virtual environment with rmvirtualenv <env_name>.

One neat option if you want to keep a virtual environment but install all packages anew is the ability to remove all third-party packages in the current virtual environment using wipeenv. You can then use pip to reinstall a package’s dependencies or a set of packages from a requirements.txt file.

Temporary Virtual Environments

Sometimes you just want to try something out quickly in a clean virtual environment, if for example you are reviewing a Pull Request and want to check installation and tests pass. virtualenvwrapper can help here as it has the mktmpenv. There are two options here -c|--cd or -n|--no-cd which changes directory post-activation or doesn’t respectively. The environment gets a unique name and will be deleted automatically when it is deactivated.

virtualenvwrapper hooks

Just like the various hooks available in Git, virtualenvwrapper also supports hooks that allow scripts to be run in response to various events. These reside under your $VIRTUALENVWRAPPER_HOOK_DIR which by default is the same as your $WORKON_HOME directory and in a typical standard installation will be ~/.virtualenvs.

The available scripts that are recognised are…

• get_env_details
• initialize
• premkvirtualenv
• postmkvirtualenv
• precpvirtualenv
• postcpvirtualenv
• preactivate
• postactivate

Each of these is a simple shell script and will start with the scripting language to use e.g. #!/usr/bin/bash or #!/usr/bin/zsh depending on your shell. You can then script the actions you wish to take when the script is executed.

Install minimal requirements

I’m a big fan of dotfiles¹, mine are hosted on GitLab, it’s a repository of my configuration files and scripts that I use regularly across multiple computers. Because I’m lazy I wrote a couple of requirements.txt files for installing packages in my virtual environments.

• requirements.txt : holds everything I might ever want to use in Python.
• python-lsp-requirements.txt : Install packages for setting up a Python Language Server (which I use from Emacs).

• venv_minimal_requirements.txt

  a minimal set of the most common Python packages I am likely to want when creating a new virtual environment.

Because I have my dotfiles cloned to the same location on every computer (~/dotfiles) I added the following to the ~/.virtualenvs/postmkvirtualenv ² which will install all of the packages listed in ~/dotfiles/python/venv_minimal_requirements.txt whenever I create a new virtual environment, whether that is with mkvirtualenv or mktmpenv.

pip install --no-cache-dir -r ~/dotfiles/python/venv_minimal_requirements.txt

This ensures the latest versions of each package listed in ~/dotfiles/python/venv_minimal_requirements.txt were downloaded and installed as the --no-cache-dir prevents using cached versions of packages.

A smarter script

This served me well for a time, but occasionally I found I didn’t want to install any packages in a new virtual environment (most often when testing new branches using mktmpenv) and I’d have to remember to comment out the line in the hook file (~/.virtualenvs/postmkvirtualenv) before creating the environment. Typically though I’d forget to do this and would have to halt installation of required packages, deactivate the environment, then comment it out and create a new environment.

This quickly became irksome.

But ~/.virtualenvs/postmkvirtualenv is just a script and so we can use a bit of scripting knowledge to make it interactive and ask the user if they want to install the packages listed in venv_minimal_requirements.txt. I found a really useful answer on StackOverflow in the How do I prompt for yes/no/cancel input in a Linux shell script that showed several different ways to prompt the user for a response as to whether they want to do something.

I therefore updated my ~/.virtualenvs/postmkvirtualenv to the following which prompts for a numeric response, 1 for Yes and 2 for No and takes the appropriate action, installing using my original invocation of pip if I want to install packages and enter 1 or installing nothing if I enter 2.

#!/usr/bin/zsh
# This hook is sourced after a new virtualenv is activated.


echo "Do you wish to install minimal requirements (from venv_minimal_requirements.txt)? "
select yn in "Yes" "No"; do
  case $yn in
    Yes ) pip install --no-cache-dir -r ~/dotfiles/python/venv_minimal_requirements.txt; break;;
    No ) echo "No packages installed. install packages with 'pip'.\n"; break;;
  esac
done

NB You may want to tweak the opening shebang if you use the Bash shell.

Drawbacks

There are a couple of drawbacks I’ve found to using using virtualenvwrapper.

The first is that mkproject doesn’t allow nesting of project directories, you have to specify a single directory and it will be created under the $PROJECT_HOME directory with the associated environment name. This doesn’t work for me as I use the structure ~/work/git as the base but then have sub-directories based on the Git Forge (GitHub/GitLab/Codeberg) the repository is associated with and further nesting to reflect the user/organisation within as I have both a personal and work accounts. E.g. ~/work/git/hub/ns-rse/ns-rse.github.io which is the source (github.com/ns-rse/ns-rse.github.io) for my blog hosted in GitHub under my work account (ns-rse) or ~/work/git/lab/nshephard/tcx2gpx which is a project of mine (tcx2gpx) which is hosted on GitLab.

This means that if I wanted to create a project with mkproject based on $PROJECT_HOME being /work/git following this structure I would specify mkproject git/lab/new_project and whilst the directory is created, the virtual environment is created as git/lab/new_project which is truncated to git and you can’t workon git because the activation scripts are nested deeper under git/lab/new_project. Further each environment I created would then conflict. I could probably work around this by creating symbolic links but in practice I just use mkvirtualenv and setvirtualenvproject after I git clone work.

This is a problem specifically of my own creation though and I can live with it/work around it.

Something other users might find causes greater friction is that virtualenvwrapper doesn’t support creating and keeping the virtual environments within the project directory itself. This is never something that I’ve wanted to do myself though as I find it tidier to keep them all in one place and easier to find and remove obsolete environments.

Conclusion

There are many options for installing and managing Python Virtual Environments including those provided by package development frameworks such as Hatch, PDM and Poetry but virtualenvwrapper is a viable option that has many neat features, more than covered here, that make working with virtual environments straight-forward. The documentation is excellent and worth reading in detail if this introductory article has sparked your interest as there are more than options than those covered here.

NB This post is based on (and hopefully improves on) two earlier blogs made by the author. Most of the improvements are Thanks to my colleagues Tamora James and Pete Heywood who reviewed a draft and provided feedback.

• virtualenvwrapper
• virtualenvwrapper hooks

Links

• Anaconda
• Hatch
• PDM
• pipx
• Poetry
• venv
• virtualenv
• virtualenvwrapper

We’ve finally put all the videos, slides and other resources together from the Best Practices in AI Afternoon event that happened on the 5th of July 2024. You can find them listed below.

Maximizing Efficiency in Large Language Models: Compute, Memory, and Fine-Tuning

Karin Sevegnani, Senior Solutions Architect, Nvidia

In this talk, we will explore the intricate balance between computational resources, memory limitations, and parameter-efficient fine-tuning techniques in large language models (LLMs). We will analyse strategies to optimize the performance of LLMs while managing these constraints effectively. From efficient memory utilization to streamlined parameter fine-tuning methods, we will discuss practical approaches to maximize the efficiency of LLMs without sacrificing performance.

• Presentation slides

Docker for Machine Learning

Ryan Daniels, University of Cambridge

Writing research software in Python presents numerous challenges to reproducibility - what version of Python is being used? What about the versions of PyTorch, Scikit Learn or Numpy? Should we use Conda, or venv, or Poetry to manage dependencies and environments? How can we control randomness? Do I have the right version of Cuda Toolkit? In principle, given the same data, and same algorithms and methodology, we should be able to reproduce the results of any given experiment to within an acceptable degree of error. Dealing with the above questions introduces significant problems to reproducing experiments in machine learning. In this talk, I would like to convince you that Docker can help alleviate almost all of these questions. Furthermore, combining Docker, git and GitHub can be a powerful workflow, helping to minimise your tech stack, and declutter your python development experience.

• Presentation slides

How do you unit test an ML model?

Wahab Kawafi, University of Bristol

Covering methods such as mock testing, simulation, experiment tracking, and dataset curation. With examples in medicine, chemistry, aerospace engineering, and LLMs.

• Presentation slides

How to make your machine learning code faster

Edwin Brown, Research Software Engineering, University of Sheffield

Practical guide to profile machine learning code to find bottlenecks and to remove these bottlenecks.

• Presentation slides
• Code repository

Nvidia Self-paced training courses and other ML resources

Denis Battistella, Higher Education and Research, Nvidia

Links to the resources discussed in the presentation:

• Training from Nvidia:
  • Deep Learning Institute and Training Solutions
  • Self-paced training courses
  • Instructor-led workshops
  • DLI University Ambassador Program
  • Deep Learning Programs for Educators
  • Twin Karmakharm (t.karmakharm@sheffield.ac.uk) of the RSE group is the University of Sheffield’s local DLI ambassador and can help with access to the above training resources.
• PhD fellowships: https://research.nvidia.com/graduate-fellowships#gf-section-graduate-fellows-
• Accelerated software:
  • Denis touched upon NGC: software repository for accelerated software, including SDKs, frameworks, containers – which you can download and deploy for free. Regularly updated.
  • New released NIM accelerated inference microservices to accelerate the deployment of Gen AI models.

Acknowledgements

We’d like to say thank you to all the speakers and attendees for making Best Practices in AI Afternoon a great success!

Thank you to Emma and Kate from the Centre for Machine Intelligence (CMI) for all the help with organising the event and the CMI and Nvidia for sponsoring the event.

Photos from the day

Karin Sevegnani presenting “Maximizing Efficiency in Large Language Models: Compute, Memory, and Fine-Tuning”

Ryan Daniels presenting “Docker for Machine Learning”

Edwin Brown presenting “How to make your machine learning code faster”

Bob Turner presenting “From Research Software to Software as a Service”

Denis Battistella presenting “Nvidia Self-paced Training Courses”

Q & A Panel with Andy Grant, Edwin Brown, Ryan Daniels, and Christopher Wild

Due to a security vulnerability, all users are advised to update their installation of R to version 4.4.0 or newer as soon as possible (ideally within the next month).

• YoYo and Managed Desktop users
• Unmanaged or Ronin users
• University of Sheffield HPC users
• FAQs
• General advice

Background

We want to bring to your attention the security risks associated with handling R data files (with the file extension .rds and .rdx). While the University has not experienced any cyber security incidents as a result of this vulnerability, it is imperative to remain vigilant and informed about potential threats and act accordingly.

A vulnerability (CVE-2024-27322) has been identified in the R language that allows attackers to run any code they want without your permission when certain types of (maliciously-crafted) data files are loaded, which could potentially lead to unauthorised access, manipulation of systems or unauthorised exporting of data. This vulnerability affects RDS (R Data Serialization) format files and RDX (R database index) files. Attackers can exploit this by crafting malicious RDS or RDX files to execute arbitrary commands on the target device. The vulnerability arises from how R implements data loading, particularly through the readRDS function, used to load RDS and RDX files. Despite expectations that loaded files don’t run code, this vulnerability allows for unauthorised code execution.

The most concerning aspects of this exploit have been fixed in R 4.4.0 (with 4.4.1 being the newest version at the time of writing) and we strongly recommend that all R users at the University of Sheffield upgrade their R version immediately. However, it is important to note that there will continue to be a risk when using RDS or RDX files from untrusted sources, regardless of whether you are using R >= 4.4.0. As always, members of the University of Sheffield should use their best judgement when running code from sources outside of the University.

YoYo and Managed Desktop users

The upgrade to R >= 4.4.0 will happen automatically in late July and you will not need to take any further action to update R itself. You will need to recreate your R package library for R >= 4.4.0 - reinstalling any R packages that you currently need/use (using e.g. update.packages()).

Unmanaged machine or Ronin users

We recommend that you have a recent backup of your files before beginning the update.

1. Go to the R project’s website
   • Choose the appropriate installer option from the list based on your machine.
2. Navigate to the precompiled package:
   • For Windows machines:
     • Choose base
     • Click Download R 4.4.1 for Windows.
     • Open the downloaded file and follow the installation instructions.
   • For Mac machines:
     • Choose the .pkg file that is appropriate for your macOS version and architecture.
     • Open the downloaded file and follow the installation instructions.
3. When the installer has completed, open R Studio. In the Tools menu, select Global Options…
4. On the General settings page, in the R Sessions section, view the file path under R Version and click Change…
5. Choose R version 4.4.1 from the pop up file explorer. Click OK.
6. Restart RStudio when prompted.
7. (Re)install the R packages you use in your work (your R package library is specific to the version of R you’re using) .
8. Test your workflows with R 4.4.1
9. Delete your older version(s) of R.
   1. To do this, open your file explorer and navigate to where your base R files are located. Use the file path from step 4 to guide you to the files if they are not obvious.
   2. Right click on the older version(s) and select Delete.
10. When all the old versions have been deleted, the upgrade task is complete.

University of Sheffield HPC users

The versions of R centrally installed on Stanage and Bessemer may be older than 4.4.0. Those installations have recently been updated to include a fix for the vulnerability.

If you’re using R via Conda on Bessemer or Stanage, see the guidance for Conda below.

FAQs

What are RDS and RDX data files for R (.rds, .rdx), and why are they used?

These are binary files used to store R objects such as data frames, lists, functions, and other R data structures. They are commonly used for saving and loading data within the R programming environment.

Do I have to upgrade to R 4.4.0 or newer?

Upgrading to R >= 4.4.0 is mandatory as it addresses recent security vulnerabilities. It’s important to remember that even with the latest version there is still a risk with untrusted RDS files.

How long will it take to upgrade?

The time to upgrade to >= 4.4.0 can range from 2 minutes to 30 mins depending on the internet speed and other factors.

How do I upgrade R versions when using Conda or Docker containers?

Upgrading R versions within Conda or Docker containers is typically straightforward due to their support for a drop-in replacement approach. However, certain considerations and best practices can streamline the process and mitigate potential complexities.

Upgrading R in Conda environments

To upgrade R in your Conda environment, you can use the following command:

conda install -c conda-forge r-base>=4.4

Significant changes, such as a compiler or low-level library update, may necessitate updates to every package in the environment. In such cases, creating a new environment based on the same specifications might offer a simpler solution.

Upgrading R in Docker containers

To upgrade R in a Docker container, ensure that the Docker image’s R version meets your requirements. Typically, you specify the desired R version in the Dockerfile, ensuring compatibility with your application’s dependencies.

Docker image r-base version >= 4.4

One potential issue arises if the older Conda environments rely on different compiler or library versions that are incompatible with the new R version or other dependencies within the environment. In such cases, addressing compatibility issues may require a solve-by-issue approach, where each problem is tackled individually. This could involve updating dependencies, modifying the environment configuration, or seeking alternative solutions depending on the specific challenges encountered.

What if my workflow is dependent on an older version of R?

We advise you to start testing your workflow on the new version, documenting any warnings or errors that might arise. You can book a Code Clinic with the Research Software Engineering (RSE) team, to discuss your specific workflow and concerns in detail and get hands-on assistance with debugging any errors or adapting your code to the new version.

Can we always trust packages downloaded from CRAN?

While CRAN offers some security measures, using a package from CRAN isn’t guaranteed to be entirely-risk free. Always use your best judgement.

What’s new in R 4.4.0?

Changes in R version 4.4.0 can be found in R News Documentation.

What if I don’t use R Studio as my default environment for R?

We recommend that you read the documentation for your operating system / chosen operating system package manager (if applicable). If you cannot find an appropriate method, then we recommend starting the update process by either uninstalling the older version of base R or (less preferable) manually deleting the base R files before downloading and installing version >= 4.4.0. The FAQs linked to from https://cloud.r-project.org/ may also be useful.

General advice

1. Use trusted sources
   • Use packages from a reputable source with a history of well-maintained packages.
   • Use only serialised (.Rds / .Rdx) objects from trusted sources; be very wary of such files shared openly on the internet.
2. Regular updates
   • Keep your R environment and packages up-to-date with the latest security patches.
3. Data minimisation
   • Store only the data essential for your analysis to reduce the risk of a data breach. If you have sensitive data, consider storing it separately in a secure location to minimise the data at risk.
4. Version control
   • Implement version control strategies to track changes and manage your codebase effectively.
5. Maintain test cases
   • Develop simple test cases that can be used to validate your code’s results to make it easier and more reliable to upgrade between versions of R and other packages.

Members of the University of Sheffield have access to a range of GPU resources for carrying out their research, available in local (Tier 3) and affiliated regional (Tier 2) HPC systems.

As of March 2024, the N8 CIR Bede Tier 2 HPC facility now includes an Open Pilot of 3 NVIDIA GH200 Nodes which are available to all users.

Each GH200 node in Bede contains a single NVIDIA GH200 Grace Hopper Superchip - a 72 core NVIDIA Grace ARM CPU connected to a single NVIDIA Hopper GPU via a 900GB/s NVIDIA NVLink-C2C interconnect. This new interconnect allows data to be moved between the host and device with a much higher bandwidth than in traditional PCI-e based systems, reducing the time spent transferring data.

The following figure shows the theoretical peak bandwidth for the range of GPU interconnect technologies used in a range of GPUs.

Source: github.com/ptheywood/gpu-interconnect-plots

PyTorch LLM Fine-tuning Benchmark

To illustrate the performance of the GH200 GPUs for machine learning workloads, a benchmark of LLM fine-tuning previously used by Farhad Allian (a Research Data Engineer in Research & Innovation IT) to investigate the performance of NVIDIA L40 GPUs for machine learning benchmarked on the GH200 GPUs in Bede.

The benchmark uses the HuggingFace Transformers run_clm.py example to train and evaluate the fine-tuning of the GPT-2 124 million parameter LLM using the WikiText-2 (raw) dataset in FP32 and FP16 precisions. Each benchmark was repeated 3 times, using a single batch size of 8. This batch size allows the benchmark to be repeated on GPUs with lower memory capacity, but larger batch sizes would likely improve performance for GPUs with sufficient memory, such as the GH200.

As of April 2024, pre-built binary wheels and conda packages for PyTorch for aarch64 systems such as the GH200 do not include CUDA support. Instead, the benchmark was containerised via Apptainer, using containers based on the NGC PyTorch containers. Version 24.02 was used for this benchmark, resulting in a software environment containing:

• Python 3.10
• CUDA 12.3.2
• PyTorch 2.3.0a0+ebedce2
• HuggingFace Transformers 4.37.0

The benchmark was then executed on V100 GPUs in Bessemer, A100 & H100 PCIe GPUs in Stanage, and the GH200 GPUs in Bede. Source files, instructions, job submission scripts and the generated results and figures can be found in the RSE-Sheffield/pytorch-transformers-wikitext2-benchmark GitHub repository.

FP32 Results

The following figures and tables show the benchmark data for FP32 training and inferencing on across a range of GPUs.

This includes the runtime training and inferencing phases in seconds (lower is better), and the samples processing rate in samples per second (higher is better).

As you might expect, newer generations of GPU offer reduced application runtimes and increased performance compared to previous generations, with the GH200 outperforming the V100 SXM2 GPUs in Bessemer, the A100 SXM4 GPUs in Stanage and the H100 PCIe GPUs in Stanage.

FP16 Results

The following figures and tables show the benchmark data for FP16 training and inferencing on across a range of GPUs.

This includes the runtime training and inferencing phases in seconds (lower is better), and the samples processing rate in samples per second (higher is better).

As with the FP32 results, the newer generations of GPU offer improved performance over older GPUs, with the GH200 out-performing the other models. The relative performance difference will vary from workload to workload, with larger batch sizes likely showing increased performance.

Accessing Bede

As a member organisation of the N8 CIR, Bede is available for use by an researchers at the University of Sheffield.

Access is granted on a per project basis, with the N8 CIR Bede website providing instructions on how to apply for access via the online form. Once submitted, the application will be reviewed and if deemed appropriate and compatible with Bede the project will be created.

Bede’s online documentation now includes GH200 specific information on the appropriate pages, in addition to the high level overview of the GH200 pilot. However, as there are only a limited number of GH200 GPUs in Bede at this time, jobs may spend a significant amount of time in the queue.

In addition to Bede, Sheffield researchers can also access a range of GPUs in our local Tier 3 facilities Bessemer and Stanage; as well as the Tier 2 JADE HPC Facility.

• Monday 18th/Tuesday 19th March 2024
• Monday 22nd/Tuesday 23rd April 2024
• Monday 20th/Tuesday 21st May 2024
• Monday 24th/Tuesday 25th June 2024

What are Git, GitHub and GitKraken?

Git is a system of version controlling your code. Think of it as a lab-book or doctors notes that are taken as you progress through your work, recording conditions, saving what has worked and correcting what doesn’t.

GitHub is a website that allows people to work collaboratively on version controlled code.

GitKraken is a client for working with Git and GitHub that includes both a GUI (Graphical User Interface) and a CLI (Command Line Interface)

Who is the course for?

Everyone who writes code! If you write scripts to analyse your code in R, Stata or Matlab you would benefit from using Git to version control your code and GitHub to share your code and make it open. If you write Python, JavaScript, C/++ code as part of a team in your research group you would benefit from using Git and GitHub to work together.

Getting started with these tools can be overwhelming but by taking this course you will be introduced to the concepts behind them and how to use them effectively to not just version control your own work but work with others on the same code.

The course material is available online if you want to take a peek and the first half using Git and publishing web-pages can be worked through in your own time. The real benefit comes from participating in the collaborative exercises in the second half where you work together on projects making Pull Requests and resolving problems that arise.

If you’ve never used Git, GitHub or GitKraken or have only just started then sign-up and come and learn more about these powerful tools.