AI Platform Engineer: Multi-Tenant LLM Orchestration (Hybrid)

Volta Software; Lombardia; Hybrid; EUR 90,000 - 130,000; Full time

Job summary

Volta Software seeks a backend engineer transitioning to AI systems to scale production-grade, multi-tenant pipelines. You will design tenant configurations and thresholds, ensuring per-customer data separation and observability across the stack.

You will optimize LLM orchestration, implement fault-tolerant queues, and drive cost-aware, secure deployments with robust testing and gradual rollout.

Qualifications

• Strong Python (FastAPI or similar), with typing and tests.
• Production LLM orchestration involving retries, tool calls and versioned prompts.
• Containerised services deployed on Cloud Run, GKE, EKS or ECS.
• Observability through tracing and metrics (Langfuse, OpenTelemetry).
• CI/CD and automated testing for non-deterministic systems.
• Fluent English; nice-to-have French or Italian.

Responsibilities

• Design the tenant configuration model (prompts, schemas, rules, feature switches).
• Implement true multi-tenancy with per-tenant data separation across services, storage, caches and traces.
• Scale the pipeline with Cloud Run components and robust queuing and retries.
• Instrument services for traceability and cost visibility across tenants.
• Control costs with per-tenant dashboards, caching, and batching of LLM workloads.
• Ensure security via guardrails, least-privilege access and CI/CD gated rollout.

Skills

Python

FastAPI

LLM orchestration

Typing

Testing

CI/CD

Containers

Cloud Run

Kubernetes

Observability

Distributed systems

Education

Bachelor's in CS

Tools

Docker

Langfuse

OpenTelemetry

CI/CD tooling

GKE

EKS

Cloud Run

Vertex AI

Job description

Volta Software seeks a backend engineer transitioning to AI systems to scale production-grade, multi-tenant pipelines. You will design tenant configurations and thresholds, ensuring per-customer data separation and observability across the stack.

You will optimize LLM orchestration, implement fault-tolerant queues, and drive cost-aware, secure deployments with robust testing and gradual rollout.