Senior Principal Machine Learning Engineer

Established Search; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Established Search seeks a Senior Principal Machine Learning Engineer to lead architecture across data platforms, ML systems and generative AI. You will drive end-to-end pipelines, model lifecycle, and production-ready QA, while mentoring teams and collaborating with stakeholders.

The role emphasizes building scalable, governance-focused platforms with strong Python/SQL skills and experience with Spark, Databricks and cloud platforms. Remote within the United States with minimal travel.

Qualifications

• 12+ years of professional software or data engineering experience, with principal-level expertise.
• Expert knowledge of distributed processing, batch and streaming pipelines and warehouse/lakehouse architectures.
• Experience with Spark, Flink and cloud-native platforms (AWS/GCP/Azure).
• Strong Python and SQL skills; experience with modern LLM systems (RAG, fine-tuning, prompt engineering).

Responsibilities

• Own the technical architecture and roadmap across data infrastructure, ML systems and generative AI capabilities.
• Mentor engineers and influence technical decisions across teams.
• Build and operate end-to-end data pipelines, model training, evaluation and deployment.
• Implement drift detection, rollback strategies and automated data contracts.
• Develop and improve LLM workflows and retrieval-augmented generation.
• Ensure governance, observability and security are embedded in platform development.

Skills

Python

SQL

Distributed processing

ML lifecycle

LLM

Data pipelines

Cloud platforms

Tools

Spark

Flink

Databricks

dbt

Kafka

MLflow

Job description

Senior Principal Machine Learning Engineer

About the Business

A healthcare technology business is using digital services and AI to help people access information, navigate care and make informed healthcare decisions. Working with employers and health plans, the business aims to improve care quality, simplify the patient experience and reduce costs.

About the Role

As Senior Principal Machine Learning Engineer, you will be the most senior individual contributor within the Data and AI function, with technical responsibility spanning data platforms, warehouses, pipelines, machine learning systems and generative AI.

You will set architectural direction, resolve complex technical challenges and deliver initiatives across multiple teams. This is a hands‑on role combining production engineering with technical strategy, mentorship and cross‑functional collaboration.

The position has no direct reports.

Key Responsibilities

Technical Architecture & Strategy

• Own the technical architecture and roadmap across data infrastructure, ML systems and generative AI capabilities.
• Identify and resolve systemic architectural issues across the Data and AI stack.
• Guide complex initiatives spanning multiple teams and systems, communicating risks and key decisions to leadership.
• Work with stakeholders to translate business requirements into technical solutions.
• Design fault‑tolerant distributed pipelines supporting batch and streaming workloads.
• Implement checkpointing, replay capabilities and schema evolution.
• Improve warehouse and lakehouse performance through partitioning, query optimisation and workload isolation.
• Establish end‑to‑end data lineage, data quality service levels and automated data contract enforcement.
• Build reliable systems across model training, evaluation, deployment and production monitoring.
• Implement drift detection, reproducible deployments and rollback strategies.
• Develop and improve LLM workflows, including retrieval‑augmented generation, fine‑tuning and prompt engineering.
• Evaluate retrieval quality and groundedness, with automated regression checks.

Engineering Standards & Technical Leadership

• Contribute directly to production code while guiding high‑level system design.
• Embed governance, quality, security and observability into platform development.
• Raise engineering standards through code reviews, design feedback and coaching.
• Mentor engineers and influence technical decisions across teams.

Experience & Qualifications

• At least 12 years of professional software or data engineering experience, including substantial experience at Principal level or above.
• Expert knowledge of distributed processing, batch and streaming pipelines, and warehouse or lakehouse architecture.
• Experience with technologies such as Spark or Flink.
• Strong ML engineering experience across the full model lifecycle.
• Practical experience with modern LLM systems, including RAG, fine‑tuning and prompt engineering.
• Strong Python and SQL skills.
• Experience building cloud‑native platforms using AWS, GCP or Azure.
• Experience optimising data platform performance and implementing governance, lineage and observability.
• A track record of leading complex technical initiatives through influence and collaboration.
• Clear written and verbal communication with technical and non‑technical stakeholders.
• Familiarity with Databricks, dbt, Kafka, MLflow or equivalent tools.
• Experience in healthcare, employee benefits or another regulated industry.
• Effective use of generative AI tools to improve engineering productivity and quality.
• Experience defining governance and observability frameworks at platform scale.
• Experience mentoring senior engineers or leading technical communities.

Location & Working Arrangement

• Remote within the United States.
• Minimal travel for occasional team meetings or company offsites.