Senior LLM Ops & Inference Platform Engineer

Datasnipper; Amsterdam; On-site; EUR 90,000 - 140,000; Full time

Job summary

DataSnipper is hiring a senior backend/platform engineer to own the LLM gateway, deploy models, and run the shared evaluation platform. You will shape production AI infrastructure used by hundreds of thousands of users, with a focus on reliability, security, and cost attribution.

You will work across Azure and GCP, implement infrastructure as code, and collaborate with product and ML engineers to build self-service platform capabilities that scale with demand.

Qualifications

• 5+ years in backend or platform engineering, with strong production Python
• Experience building or running LLM inference infrastructure: a gateway or routing layer with multiple providers, failover, rate limiting and cost attribution
• Experience with cloud at the infrastructure level and infrastructure as code we run across Azure and GCP with Terraform
• Experience running a shared service in production: on-call, incidents, postmortems, SLOs
• Hands-on experience with observability tools (OpenTelemetry, Grafana), including instrumenting services and designing dashboards and alerts
• Comfort with privacy and compliance work: PII handling, anonymisation, retention, access control
• Experience building platform or shared-service capabilities consumed by multiple internal teams

Responsibilities

• Own the LLM gateway: routing, provider failover, rate limits, retries, and cost attribution across multiple model providers
• Deploy, version and deprecate models across clouds, regions and environments, including quota and capacity planning, managed as infrastructure as code
• Build the shared evaluation platform: versioned datasets, experiment tracking, run and result schemas, reporting, and trace linkage back to the run
• Own the infrastructure for async and long-running AI workloads
• Own observability for AI traffic: latency, retries and fallbacks, token usage, cost and errors, per team and per use case
• Take part in the on-call rotation, run incidents, and close the follow-ups
• Implement the security and compliance controls the platform is held to: retention, access control, RBAC and SSO
• Define and maintain clean integration contracts between the platform and the teams that consume it
• Partner with product and ML engineers to turn their requirements into platform capabilities that are self-service rather than a request queue

Skills

Backend engineering

Python (production)

LLM inference infra

Cloud infrastructure

Infrastructure as code

Observability tooling

On-call & incidents

Privacy & compliance

Tools

Terraform

OpenTelemetry

Grafana

Azure

GCP

Job description

DataSnipper is hiring a senior backend/platform engineer to own the LLM gateway, deploy models, and run the shared evaluation platform. You will shape production AI infrastructure used by hundreds of thousands of users, with a focus on reliability, security, and cost attribution.

You will work across Azure and GCP, implement infrastructure as code, and collaborate with product and ML engineers to build self-service platform capabilities that scale with demand.