AI Engineer | Public Sector Consulting

IBM; Boston (MA); Hybrid; USD 106,000 - 159,000; Full time

Job summary

IBM Consulting FutureNow Center is seeking an experienced Senior AI Engineer to design, build, and deploy enterprise AI solutions across client environments, leveraging LLMs and RAG. You will collaborate with Architects, Data Scientists, Engineers, and stakeholders to create secure, production‑ready AI systems and to mitigate hallucinations while improving retrieval quality.

This hybrid role offers mentorship, continuous learning, and opportunities to drive governance and responsible AI

Qualifications

• Hands-on experience with LLMs and LLM APIs.
• Production RAG solutions and/or agent-based AI systems.
• Strong API development and systems integration experience.
• Experience deploying AI-enabled applications in production.
• Knowledge of AI evaluation, hallucination mitigation, and responsible AI.

Responsibilities

• Design, build, and deploy AI-powered applications using LLMs and modern frameworks.
• Deliver enterprise AI solutions including RAG, agentic workflows, and semantic search.
• Integrate AI capabilities into existing platforms via APIs and integration patterns.
• Collaborate with Architects, Data Scientists, Engineers, and stakeholders.
• Evaluate and optimize AI system performance and user experience.
• Mitigate hallucinations and improve retrieval quality.
• Contribute to AI governance, security controls, and responsible AI practices.
• Translate business requirements into production-ready technical solutions.
• Mentor and share AI engineering best practices across teams.

Skills

LLMs

RAG

Agent-based AI

API development

AI governance

Embeddings

Vector search

Stakeholder engagement

Production deployment

Tools

LangChain

LangGraph

DSPy

Semantic Kernel

LlamaIndex

Qdrant

Pinecone

Weaviate

FAISS

CI/CD

Containerisation

Job description

A career in IBM Consulting is built on long-term client relationships and close collaboration worldwide. You’ll work with leading companies across industries, helping them shape their hybrid cloud and AI journeys. With support from our strategic partners, robust IBM technology, and Red Hat, you’ll have the tools to drive meaningful change and accelerate client impact. At IBM Consulting, curiosity fuels success. You’ll be encouraged to challenge the norm, explore new ideas, and create innovative solutions that deliver real results. Our culture of growth and empathy focuses on your long-term career development while valuing your unique skills and experiences.

In this role, you'll work in one of our IBM Consulting FutureNow Centers, where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology.

Your role and responsibilities

Join IBM Consulting FutureNow's Hybrid Cloud & Data practice and help clients unlock the value of Artificial Intelligence at scale.

We're looking for an experienced AI Engineer to design, build, and deploy enterprise AI solutions across a range of client environments. Working alongside Architects, Data Scientists, Engineers, and client stakeholders, you'll help translate emerging AI capabilities into practical, secure, and production-ready solutions.

This role is particularly well suited to individuals who enjoy solving complex technical challenges through Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), agentic AI systems, and modern integration patterns.

You'll work across a variety of high-profile programmes, helping organisations operationalise AI responsibly while balancing scalability, governance, security, and business outcomes.

Role & Responsibilities

As a Senior AI Engineer, you will play a key role in the design, development, and deployment of AI-powered solutions across client engagements.

Core Responsibilities

• Design, build, and deploy AI-powered applications and services using Large Language Models and modern AI frameworks.
• Deliver enterprise AI solutions including Retrieval-Augmented Generation (RAG), agentic workflows, semantic search, and knowledge retrieval systems.
• Integrate AI capabilities into existing business platforms, products, and services through APIs and modern integration patterns.
• Collaborate with Architects, Data Scientists, Engineers, and stakeholders to shape scalable and secure AI solutions.
• Evaluate and optimise AI system performance, reliability, and user experience.
• Develop approaches to mitigate hallucinations, improve retrieval quality, and enhance overall model outcomes.
• Contribute to AI governance, evaluation frameworks, security controls, and responsible AI practices.
• Translate business requirements into production-ready technical solutions.
• Support mentoring, knowledge sharing, and AI engineering best practice across delivery teams.

Required education

None

Required technical and professional expertise

• Hands‑on experience working with Large Language Models (LLMs) and LLM APIs.
• Experience implementing production RAG (Retrieval‑Augmented Generation) solutions and/or agent‑based AI systems.
• Strong API development and systems integration experience.
• Experience designing and deploying AI‑enabled applications in production environments.
• Understanding of AI evaluation techniques, hallucination mitigation, prompt engineering, and responsible AI principles.
• Experience working with unstructured data, embeddings, vector search, or semantic retrieval approaches.
• Strong stakeholder engagement and communication skills.
• Experience delivering solutions from concept through to deployment and adoption.

This role is subject to pre‑employment screening in line with the UK Government's Baseline Personnel Security Standard (BPSS). Additional National Security Vetting (NSV) requirements may apply, including eligibility for Security Check (SC) or Developed Vetting (DV).

Preferred technical and professional experience

• Experience with frameworks such as LangChain, LangGraph, DSPy, Semantic Kernel, LlamaIndex, or similar.
• Experience with vector databases such as Qdrant, Pinecone, Weaviate, FAISS, or equivalent.
• Familiarity with agent orchestration, tool calling, and multi‑agent architectures.
• Experience deploying AI solutions within cloud environments (AWS, Azure, or GCP).
• Exposure to AI governance, model assurance, or regulated industries.
• Experience operating within Public Sector, Defence, Financial Services, or other high‑assurance environments.
• Familiarity with CI/CD, containerisation, and modern software engineering practices.

ABOUT BUSINESS UNIT

IBM Consulting is IBM’s consulting and global professional services business, with market leading capabilities in business and technology transformation. With deep expertise in many industries, we offer strategy, experience, technology, and operations services to many of the most innovative and valuable companies in the world. Our people are focused on accelerating our clients’ businesses through the power of collaboration. We believe in the power of technology responsibly used to help people, partners and the planet.

YOUR LIFE @ IBM

In a world where technology never stands still, we understand that, dedication to our clients success, innovation that matters, and trust and personal responsibility in all our relationships, lives in what we do as IBMers as we strive to be the catalyst that makes the world work better.

Being an IBMer means you’ll be able to learn and develop yourself and your career, you’ll be encouraged to be courageous and experiment everyday, all whilst having continuous trust and support in an environment where everyone can thrive whatever their personal or professional background.

Our IBMers are growth minded, always staying curious, open to feedback and learning new information and skills to constantly transform themselves and our company. They are trusted to provide on‑going feedback to help other IBMers grow, as well as collaborate with colleagues keeping in mind a team focused approach to include different perspectives to drive exceptional outcomes for our customers. The courage our IBMers have to make critical decisions everyday is essential to IBM becoming the catalyst for progress, always embracing challenges with resources they have to hand, a can‑do attitude and always striving for an outcome focused approach within everything that they do.

Are you ready to be an IBMer?

ABOUT IBM

IBM’s greatest invention is the IBMer. We believe that through the application of intelligence, reason and science, we can improve business, society and the human condition, bringing the power of an open hybrid cloud and AI strategy to life for our clients and partners around the world.

Restlessly reinventing since 1911, we are not only one of the largest corporate organizations in the world, we’re also one of the biggest technology and consulting employers, with many of the Fortune 500 companies relying on the IBM Cloud to run their business.

At IBM, we pride ourselves on being an early adopter of artificial intelligence, quantum computing and blockchain. Now it’s time for you to join us on our journey to being a responsible technology innovator and a force for good in the world.

IBM is proud to be an equal‑opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, genetics, pregnancy, disability, neurodivergence, age, or other characteristics protected by the applicable law. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.

OTHER RELEVANT JOB DETAILS

IBM wants you to bring your whole self to work and for you this might mean the ability to work flexibly. If you are interested in a flexible working pattern, please talk to our recruitment team to find out if this is possible in the current working environment.

Job Title

Job ID

133853

City / Township / Village

State / Province

Hampshire, London, City of, Manchester, Leicester

Country

United Kingdom

Work arrangement

Hybrid

Area of work

Data & Analytics

Employment type

Regular

Position type

Professional

Up to 40% or 2 days a week (home on weekends- based on project requirements)

Company

Shift

General (daytime)

Is this role a commissionable/sales incentive based position?