Senior AI Engineer — Remote, LLM Systems & Evaluation

Mapbox; United Kingdom; Hybrid; GBP 100,000 - 150,000; Full time

Benefits offered by this job

Hybrid/Remote options

Private health and dental coverage

Paid time off & holidays

Volunteer time off

Job summary

Mapbox is seeking an experienced AI/Software Engineer to own the design and delivery of multi-component AI systems. You will own MVPs, build data pipelines, and evaluate external data and benchmarks to guide when to adopt existing solutions.

You will work across teams to ship high-quality features with scalability and latency goals in mind. You will design evaluation frameworks for non-deterministic outputs, craft measurement strategies, and ensure on-call readiness while collaborating across

Qualifications

• BS in a STEM field with 5+ years of software engineering experience and production ownership.
• 2+ years shipping LLM-backed features in production with error budgets and on-call rotations.
• Experience designing evaluations for non-deterministic systems and datasets.
• Deep SQL and at least one distributed processing framework.
• Fluency in Python or TypeScript and reading multi-language codebases.
• Experience diagnosing latency in distributed request paths.
• Comfort with ambiguity and shipping narrow, working solutions.

Responsibilities

• Define product designs, build measurement frameworks, and gate changes on regression results.
• Run continuous evaluation of APIs, SDKs, and data representations from the end-user perspective.
• Own the MVP against a north-star design, balancing speed and technical quality.
• Build data pipelines for ingestion, conflation, and entity resolution with quality checks.
• Track external datasets and benchmarks to decide when to adopt existing solutions.
• Design feedback loops so product usage creates data to improve it.
• Define the model vs. tool boundary and manage the model harness.
• Target latency and cost per request with caching and routing strategies.
• Create internal tooling and harnesses (CLI, MCP) to accelerate iteration.
• Raise the bar through code and design reviews and evangelize eval practices.
• Participate in on-call rotations to keep systems available 24/7.

Skills

LLM-backed features

Python

TypeScript

SQL

Data engineering

Agent orchestration

Tool calling

Distributed systems

On-call ownership

Education

Bachelors degree in STEM

Tools

MCP

Eval harness

CI/CD pipelines

Job description

Mapbox is seeking an experienced AI/Software Engineer to own the design and delivery of multi-component AI systems. You will own MVPs, build data pipelines, and evaluate external data and benchmarks to guide when to adopt existing solutions.

You will work across teams to ship high-quality features with scalability and latency goals in mind. You will design evaluation frameworks for non-deterministic outputs, craft measurement strategies, and ensure on-call readiness while collaborating across