Staff ML Platform Engineer: Build Production ML Pipelines

Datavant; Indianapolis (IN); In loco; USD 224.000 - 280.000; Tempo pieno

Descrizione del lavoro

Datavant is seeking a Staff ML Platform Engineer to lead the Data & ML Platform team. You will drive the paved road from config to production, enabling Data Science to self-serve with reliable training, inference, and observability.

You will own architecture for LLM-endpoint serving, manage CI/CD spine, and ensure safe handling of PHI. Strong AWS, Spark, and ML tooling depth are required.

Competenze

• 10+ years of software engineering experience, with 3+ years designing, evolving, and operating enterprise-scale ML platforms in production.
• Strong technical judgment under ambiguity and a track record of setting standards across teams.
• Hands-on production experience with Databricks and/or Amazon SageMaker, MLflow, and at least one core ML framework (PyTorch, TensorFlow, or similar).
• Fluency in Java (or a JVM) and Python, with depth in Apache Spark for large-scale data and distributed compute.
• Real depth in AWS: networking, IAM, GPU compute, and the storage and messaging services this role touches.

Mansioni

• Set technical direction across ML training, serving, and observability, and be the escalation point for infrastructure problems (GPU capacity, Spark tuning, production incidents).
• Own and evolve our paved-road framework (CI/CD spine, model-workflow scaffolding, Databricks Asset Bundles) for self-serve production workflows.
• Lead architecture for LLM-endpoint serving across Databricks, AWS, Snowflake and self-hosted deployments, covering latency, cost, caching, evaluation, and PHI-safe routing.
• Own the standards and tooling for MLflow, model registry, training image supply chain, and observability across training and inference.
• Partner with Data & ML Platform teammates to present a cohesive ML platform to Data Science, App Dev, and Operations teams at Datavant.
• Serve as a key technical input to vendor and platform selection across model providers, ML tooling, and observability.
• Mentor senior engineers, provide guidance to platform consumers, and stay hands-on with IaC.

Conoscenze

ML platform design

Databricks

SageMaker

MLflow

PyTorch

TensorFlow

Java

Python

Apache Spark

Terraform

Kubernetes

CI/CD

AWS

GPU compute

cost optimization

Strumenti

Databricks Asset Bundles

Unity Catalog

Iceberg/Delta

Terraform

Kubernetes

GitHub CI/CD

Descrizione del lavoro

Datavant is seeking a Staff ML Platform Engineer to lead the Data & ML Platform team. You will drive the paved road from config to production, enabling Data Science to self-serve with reliable training, inference, and observability.

You will own architecture for LLM-endpoint serving, manage CI/CD spine, and ensure safe handling of PHI. Strong AWS, Spark, and ML tooling depth are required.