Lead ML Infrastructure & AI Platform Engineer

Harrison Clarke; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Competitive compensation

Meaningful equity

Autonomy and ownership

Job summary

Harrison Clarke in San Francisco is seeking a Lead Data & ML Infrastructure Engineer to own the production backbone for AI systems at scale. The role focuses on pipelines, platforms, and observability that convert experiments into reliable production results.

You will design systems where models monitor themselves, adapt to drift in real time, and decisions shape AI capabilities. High autonomy and impact are expected.

Qualifications

• 5+ years of production ML and data infrastructure experience.
• Experienced in modern data infra stacks: Spark, Delta Lake, Kafka, Flink, Airflow.
• Proficient with ML lifecycle platforms (MLflow, Kubeflow, SageMaker).
• Strong in observability, data quality, and pipeline SLAs.

Responsibilities

• Architect and build next-gen ML pipelines that self-heal and auto-retrain on drift.
• Design the data infrastructure layer for an AI-native stack with real-time and batch processing.
• Build production-grade ML observability for model health and drift.
• Push real-time feature serving and streaming computation.
• Own infrastructure reliability: graceful degradation and zero-downtime deployments.
• Bridge to LLM/agentic infra: model serving, inference optimization, retrieval-augmented generation.

Skills

Distributed systems

Production ML

Observability

Technical communication

Tools

Spark

Delta Lake

Kafka

Flink

Airflow

MLflow

Kubeflow

SageMaker

Job description

Harrison Clarke in San Francisco is seeking a Lead Data & ML Infrastructure Engineer to own the production backbone for AI systems at scale. The role focuses on pipelines, platforms, and observability that convert experiments into reliable production results.

You will design systems where models monitor themselves, adapt to drift in real time, and decisions shape AI capabilities. High autonomy and impact are expected.