Senior ML Platform Engineer, Production-Grade Pipelines

Datavant; City of Albany (NY); On-site; USD 224,000 - 280,000; Full time

Job summary

Datavant is seeking a Staff ML Platform Engineer to lead the ML Platform team, guiding training, serving, and observability across AWS, Databricks, and SageMaker. You will shape the paved road from config to production, enabling data science teams to self-serve while safeguarding PHI and sensitive data.

You will mentor engineers, own architecture for LLM-endpoint serving, and collaborate with Data & ML Platform teammates to deliver a cohesive platform.

Qualifications

• 10+ years of software engineering experience, 3+ years in enterprise-scale ML platforms.
• Strong technical judgment and ability to set standards across teams.
• Hands-on experience with Databricks and/or SageMaker, MLflow, and at least one core ML framework (PyTorch or TensorFlow).
• Fluency in Java and Python with deep Apache Spark knowledge.
• AWS expertise including networking, IAM, GPU compute, and storage/messaging services.
• Fluency with Terraform, containers, Kubernetes, and GitHub-based CI/CD for ML workloads.
• Direct experience serving LLMs in production, including cost management and safe handling of prompts.
• AI-native working style with tools like Claude Code, Cursor, Copilot.
• Clear written and verbal communication in async, remote settings.

Responsibilities

• Set technical direction across ML training, serving, and observability; be the final escalation point for infrastructure issues.
• Own and evolve the paved-road framework (CI/CD spine, model-workflow scaffolding) for production workflows.
• Lead architecture for LLM-endpoint serving across providers and self-hosted deployments; address latency, cost, caching, evaluation, PHI-safe routing.
• Own standards and tooling for MLflow, model registry, and training image supply chain.
• Partner with Data & ML Platform teammates to present a cohesive ML platform to DS, App Dev, and Operations teams.
• Serve as a key technical input to vendor and platform selection decisions across model providers and tooling.
• Mentor senior engineers, provide guidance to platform consumers, and contribute hands-on code and IaC.

Skills

ML Platform

AWS

Java

Python

Spark

Terraform

Kubernetes

GitHub CI/CD

LLMs in production

Communication

Tools

Databricks

SageMaker

MLflow

Databricks Asset Bundles

Job description

Datavant is seeking a Staff ML Platform Engineer to lead the ML Platform team, guiding training, serving, and observability across AWS, Databricks, and SageMaker. You will shape the paved road from config to production, enabling data science teams to self-serve while safeguarding PHI and sensitive data.

You will mentor engineers, own architecture for LLM-endpoint serving, and collaborate with Data & ML Platform teammates to deliver a cohesive platform.