Senior Harness Engineer: AI Verification & Quality Systems

FICO; Belfast City District; On-site; GBP 85,000 - 120,000; Full time

Job summary

FICO is seeking a Senior Harness Engineer to own end-to-end harness subsystems, setting standards and guiding engineering teams as they turn raw model capability into production-grade systems. You will design, build, and maintain the verification harnesses, feedback loops, and guardrails that ensure safe, high-quality AI-generated output.

The role emphasizes hands-on development, instrumentation, and cross-team collaboration to raise reliability and drive continuous improvement in AI-assisted

Qualifications

• Bachelor's/Master's in Computer Science or related disciplines, or relevant commercial experience in software architecture, design, development, and testing.
• Seasoned software engineer with experience in large, complex codebases and a strong foundation in architecture and design; you care deeply about testing and maintainability.
• Hands-on experience with AI coding agents (e.g. Claude Code, Codex, or similar) and a well-developed feel for where they succeed and fail.
• Proven ability to build engineering tooling across a modern stack - linters and static analysis, CI/CD pipelines, containerised build/test environments, and instrumentation/observability - plus familiarity with agent instruction conventions such as AGENTS.md.
• Experience with spec-driven development, context engineering, agent orchestration, fitness functions, and developer-platform work.
• A systems mindset - you'd rather fix the environment than fix one output - and the ability to encode "what good looks like" into mechanical, repeatable rules.
• Judgement about when to reach for deterministic, computational controls vs LLM-based ones - and an understanding of cost, speed, and reliability trade-offs.
• Experience owning quality-gating processes and defining release criteria to ensure engineering standards are consistently met.
• Working knowledge of the security surface unique to autonomous agents - prompt injection, tool/permission scoping, sandboxed execution, and audit trails for agent actions
• Experience with consumer/contract testing approaches (e.g. Pact) to validate service integrations across distributed systems.
• Excellent communication skills; able to articulate design with architects and drive standards across teams.

Responsibilities

• Design, build, deploy, and support core components of the harness - the guides, feedback loops, guardrails, and shared context that turn raw model capability into production-grade engineering.
• Own and evolve feedforward guides - agent instruction files, reusable skills, architectural rules, reference docs, and codemods - and drive team-wide standardisation so agents get it right the first time.
• Build feedback sensors - custom linters, static analysis, structural and architecture-fitness tests, verification loops, and LLM-as-judge reviewers - that catch issues automatically before they reach human reviewers.
• Own quality gating and release criteria for agent-produced work, defining authority boundaries for what agents may merge unaided and the escalation rules for what must route to a human.
• Establish LLM testing infrastructure and evaluation approaches that ensure AI-generated output meets quality and safety thresholds; apply consumer/contract testing where service integration reliability matters.
• Run the steering loop - when an agent repeats a mistake, engineer a control so it can't happen again and treat repository knowledge as the system of record.
• Decide where each control runs in the path to production - fast checks pre-commit, more expensive checks post-integration, and continuous sensors that scan for drift outside the change lifecycle.
• Improve observability into agent work and track the measures that matter - cost per merged PR, time-to-merge for agent-assisted PRs, review velocity relative to PR size, defect escape rate, and agent-PR survival rate.
• Partner with product and platform teams to turn specifications and acceptance criteria into enforceable controls.
• Serve as a source of technical expertise and mentor engineers across teams in harness practices and the effective, responsible use of AI tools.

Skills

Software engineering

AI coding agents

Testing and maintainability

Observability/instrumentation

Security guardrails

Mentoring

Education

Bachelor's/Master's in CS or related

Tools

CI/CD pipelines

Containerised build/test

Linters/static analysis

Pact/contract testing

AGENTS.md familiarity

Job description

FICO is seeking a Senior Harness Engineer to own end-to-end harness subsystems, setting standards and guiding engineering teams as they turn raw model capability into production-grade systems. You will design, build, and maintain the verification harnesses, feedback loops, and guardrails that ensure safe, high-quality AI-generated output.

The role emphasizes hands-on development, instrumentation, and cross-team collaboration to raise reliability and drive continuous improvement in AI-assisted