Staff Software Engineer – ML Inference Runtime

Arm Limited; Galway; Hybrid; EUR 90,000 - 130,000; Full time

Job summary

Arm Limited is seeking engineers to help shape the future of ML and AI on Arm platforms. You will develop and optimize software that connects ML frameworks and runtimes to Arm hardware acceleration, including llama.cpp integrations, and lead projects from design to delivery.

The role requires strong skills in C++, C, and Python, plus experience with multi-team software delivery, Git/GitLab/CI, and LLM concepts. A collaborative, fast-paced environment awaits.

Qualifications

• Proven ability to deliver software features across multiple teams.
• Experience with Git, GitLab, CI, and automated tests.
• Strong analytical and problem-solving skills; able to lead significant projects end-to-end.
• Knowledge of LLM inference concepts such as tokenization, attention, KV caches, batching, and quantisation.

Responsibilities

• Develop and optimize C++, C, and Python software that integrates ML platforms with Arm hardware acceleration.
• Design and implement integrations between ML frameworks, runtimes, and hardware acceleration technologies.
• Analyze and improve performance of ML and LLM workloads.
• Lead significant work from design through delivery; mentor junior engineers; support technical decisions.

Skills

C++

Python

Software design

Software delivery across teams

Git, GitLab, CI, tests

Technical leadership

LLM inference basics

Tools

llama.cpp

Job description

Job Overview:

We have a phenomenal opportunity to help shape the future of Machine Learning and Artificial Intelligence on Arm platforms. Arm technology is used wherever computing matters. Are you passionate about ML systems and keen to influence how generative AI and other machine-learning workloads run efficiently on next-generation hardware?

Then we should talk!

We are growing our ML framework integrations team and seek dedicated and motivated engineers to join us. You will work with teams across Arm and with leading technology companies building products based on Arm technologies. You will share ideas, learn from outstanding engineers, and make contributions that have a real impact. This role offers the opportunity to provide develop the software that connects widely used machine-learning frameworks and runtimes to Arm acceleration technologies.

Responsibilities:

Responsibilities include developing and optimizing C++, C, and Python software that integrates machine-learning platforms and execution environments with Arm hardware acceleration. This work entails integration with open-source LLM software such as llama.cpp. You will design and implement integrations between ML frameworks, runtimes, and hardware-acceleration technologies, as well as analyse and improve the performance of ML and LLM workloads. The role also involves leading significant work from design through delivery, mentoring junior engineers, and supporting technical decisions.

We are looking for someone with strong analytical and problem-solving skills who enjoys finding innovative solutions to complex technical challenges. You will be comfortable working in a fast-paced environment and collaborating across complementary teams to achieve shared goals.

Required Skills and Experience :

• Strong C++, C, and Python programming and software-design skills
• Experience delivering significant software features across multiple teams
• Familiarity with Git, GitLab, continuous integration, and automated testing
• Effective technical leadership and communication skills
• Knowledge of LLM inference, including tokenization, attention, KV caches, batching and quantisation

“Nice To Have” Skills and Experience :

• Experience with llama.cpp or other open-source ML or LLM runtimes
• Experience with profiling and optimizing ML workloads
• Experience with compiler or ML graph technologies
• Experience with Vulkan, GPU compute, or other hardware-acceleration APIs
• Experience using AI-assisted software-development tools effectively

In Return:

All Arm employees are provided with vital training to succeed in their respective roles, as well as a friendly and high-performance working environment.

You will be working with a bunch of enthusiastic and brilliant colleagues. We are proud to have a set of behaviours that reflect our DEI (Diversity, Equity & Inclusion) culture and guide our decisions, defining how we work together to shape extraordinary!

Please note that no relocation package is available for this role.

|Develop C++, C and Python integrations connecting ML frameworks and LLM runtimes like llama.cpp with Arm acceleration technologies, optimising AI workloads for next-generation hardware.!

Accommodations at Arm

At Arm, we want to build extraordinary teams. If you need an adjustment or an accommodation during the recruitment process, please email [e-mail]. To note, by sending us the requested information, you consent to its use by Arm to arrange for appropriate accommodations. All accommodation or adjustment requests will be treated with confidentiality, and information concerning these requests will only be disclosed as necessary to provide the accommodation. Although this is not an exhaustive list, examples of support include breaks between interviews, having documents read aloud, or office accessibility. Please email us about anything we can do to accommodate you during the recruitment process.

Hybrid Working at Arm

Arm’s approach to hybrid working is designed to create a working environment that supports both high performance and personal wellbeing. We believe in bringing people together face to face to enable us to work at pace, whilst recognizing the value of flexibility. Within that framework, we empower groups/teams to determine their own hybrid working patterns, depending on the work and the team’s needs. Details of what this means for each role will be shared upon application. In some cases, the flexibility we can offer is limited by local legal, regulatory, tax, or other considerations, and where this is the case, we will collaborate with you to find the best solution. Please talk to us to find out more about what this could look like for you.

Equal Opportunities at Arm

Arm is an equal opportunity employer, committed to providing an environment of mutual respect where equal opportunities are available to all applicants and colleagues. We are a diverse organization of dedicated and innovative individuals, and don’t discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.