Lead ML Platform Engineer: LLM Serving & Pipelines

Hidden Jobs; United States; Remote; USD 224,000 - 280,000; Full time

Job summary

Hidden Jobs seeks a Staff ML Platform Engineer to guide the roadmap for ML training, serving, and observability within a health-data environment. You will own paved-road pipelines, CI/CD scaffolding, and architecture for LLM serving across providers, while ensuring secure handling of sensitive prompts.

You will mentor senior engineers, contribute hands-on code, and collaborate with cross-functional teams to deliver a coherent ML platform experience across data science and operations.

Qualifications

• 10+ years of software engineering experience, with 3+ years designing enterprise-scale ML platforms in production.
• Experience with at least one major managed ML platform (Databricks and/or SageMaker).
• Proficiency in Python and a JVM language, with deep Spark experience.
• Familiarity with AWS networking, GPU compute, storage, and messaging.
• Fluency with Terraform, containers, Kubernetes, and GitHub-based CI/CD for ML workloads.
• Strong communication skills for remote, cross-functional collaboration.

Responsibilities

• Set technical direction across ML training, serving, and observability, and act as escalation point for infrastructure problems.
• Own and evolve the paved-road framework with CI/CD and model-workflow scaffolding for shipping workflows from config files.
• Lead architecture for LLM endpoint serving across managed providers and self-hosted deployments, addressing latency, cost, and safe routing of prompts.
• Set standards for experiment tracking, model registries, training image supply chains, and observability across training and inference.
• Mentor senior engineers and stay hands-on writing high-leverage code and infrastructure as code.
• Collaborate with adjacent platform, application, and operations teams to deliver a coherent ML platform experience.

Skills

Python

Java

Apache Spark

PyTorch

TensorFlow

AWS

Terraform

Kubernetes

CI/CD

Observability

Tools

Databricks

SageMaker

Apache Spark

Job description

Hidden Jobs seeks a Staff ML Platform Engineer to guide the roadmap for ML training, serving, and observability within a health-data environment. You will own paved-road pipelines, CI/CD scaffolding, and architecture for LLM serving across providers, while ensuring secure handling of sensitive prompts.

You will mentor senior engineers, contribute hands-on code, and collaborate with cross-functional teams to deliver a coherent ML platform experience across data science and operations.