ML Backend Engineer: Scalable Inference for Foundations

Prior Labs GmbH; North Merrick (NY); Hybrid; USD 160,000 - 210,000; Full time

Job summary

Prior Labs GmbH is seeking an experienced backend infrastructure engineer to own the inference serving path and build scalable production systems for TabPFN-era foundation models. You will shape the core backend, design robust APIs, and manage cloud infrastructure with a strong emphasis on reliability and cost-efficiency.

You will collaborate with AI researchers to translate cutting-edge models into scalable production systems, with opportunities to specialize in inference optimization, backend

Qualifications

• 3+ years of professional experience in machine learning and backend engineering, with a proven track record of managing production infrastructure.
• Proven experience deploying and operating machine learning models in production, with strong understanding of model behavior under real traffic.
• Strong, hands-on experience designing and building production-grade APIs and backend services.
• Significant experience with cloud platforms, with GCP preferred.
• Strong experience with Infrastructure as Code (IaC) using Terraform.
• Significant experience with containerization and orchestration (Docker, Kubernetes).
• Proficiency in Python.

Responsibilities

• Own the serving path end-to-end: latency, throughput, batching, memory behavior on large inputs, and cost per prediction.
• Architect and design robust, scalable backend systems and production-grade APIs for serving foundation models.
• Build and implement high-quality, maintainable core backend services.
• Own infrastructure design, deployment, and management on cloud platforms with focus on reliability, monitoring, observability, and cost-efficiency.
• Ensure secure, GDPR-compliant data handling, including storage, access control, usage tracking, and quota management.
• Champion best practices for testing, CI/CD, documentation, and security within the engineering team.

Skills

ML backend infra

Python

Terraform

Docker

Kubernetes

GCP

API design

CI/CD

Tools

PyTorch

FastAPI

Git

Job description

Prior Labs GmbH is seeking an experienced backend infrastructure engineer to own the inference serving path and build scalable production systems for TabPFN-era foundation models. You will shape the core backend, design robust APIs, and manage cloud infrastructure with a strong emphasis on reliability and cost-efficiency.

You will collaborate with AI researchers to translate cutting-edge models into scalable production systems, with opportunities to specialize in inference optimization, backend