Principal Machine Learning Engineer

cw; United States; On-site; USD 180,000 - 260,000; Full time

Job summary

cw is seeking a Principal ML Engineer to build and maintain AI/ML systems for the business. This role blends deep technical execution with leadership, tackling complex ML problems, designing data architectures, MLOps and LLMOps, and raising engineering standards across the Data Organization.

Reporting to the Senior Director, Data Science & AI, you will mentor engineers, define architectural principles, and work primarily with Databricks on Azure, while evaluating additional data technologies as

Qualifications

• 10+ years building software or ML systems, incl. 6+ years in production ML.
• Senior-level engineer with architectural influence and standards setting.
• Strong software fundamentals: version control, testing, code review.

Responsibilities

• Shape end-to-end AI architectures, development, and deployment for business use cases.
• Design and develop agentic AI systems including multi-agent orchestration and autonomous tasks.
• Set standards for data prep, features, model selection, evaluation, and documentation.
• Partner with data engineering to ensure robust, scalable pipelines for training and inference.

Skills

Python

SQL

Databricks

Azure

MLOps

LLMOps

AI governance

Production ML

Tools

Databricks Platform

MLflow

Model Serving

Job description

Job Title

Principal Machine Learning Engineer

Job Description Summary

We are seeking a Principal ML Engineer to build and maintain artificial intelligence and machine learning systems for our business. This role combines deep technical execution with broad influence: tackling the most complex machine learning problems, designing and implementing data architectures/models, MLOps and LLMOps, and monitoring and optimization as key software, while raising the bar for engineering excellence across the Data Organization.

Job Description

We are seeking a Principal ML Engineer to build and maintain artificial intelligence and machine learning systems for our business. This role combines deep technical execution with broad influence: tackling the most complex machine learning problems, designing and implementing data architectures/models, MLOps and LLMOps, and monitoring and optimization as key software, while raising the bar for engineering excellence across the Data Organization.

Role Summary

Reporting to the Senior Director, Data Science & AI, the Principal ML Engineer is expected to move fluidly across project/product teams - embedding in projects for days or months at a time to unblock, accelerate, and uplift delivery. They will mentor other engineers at every level, push standards for technical excellence, and help establish the ML engineering, and architectural principles that guide our work. The primary platform is Databricks and the Azure cloud ecosystem, with the expectation of evaluating and adopting additional data technologies as the platform evolves.

Key Responsibilities

AI/ML Model Development

• Demonstrable experience shaping the end-to-end architecture, development, and deployment of AI, including machine learning, GenAI, and Agentic models that are tailored to business use cases.
• Design and develop agentic AI systems - including multi-agent orchestration, tool-use, evaluation, and autonomous task-execution pipelines - to automate complex enterprise workflows.
• Enforce model development standards encompassing data preprocessing, feature engineering, model selection, hyperparameter tuning, evaluation, and documentation.
• Partner with data engineering teams to ensure robust, scalable, and high-quality data pipelines that support model training and inference.

AI Operations (MLOps / LLMOps)

• Implement the organization's MLOps and LLMOps capabilities, including CI/CD pipelines for data wrangling, feature engineering, model training, evaluation, deployment, and monitoring.
• Help define and enforce standards for model versioning, experiment tracking, reproducibility, and model registry management.
• Implement robust model monitoring frameworks to detect performance degradation, data drift, concept drift, and bias in production systems, with automated alerting and retraining triggers.

AI Governance

• Implement AI governance frameworks, including policies for model risk management, fairness, explainability, privacy, and security.
• Champion sound AI principles across the global business, ensuring that human oversight and accountability are embedded in every stage of the AI development lifecycle.

Essential Skills, Knowledge & Experience

• 10+ years building software or machine learning systems, including 6+ years shipping and operating machine learning in production - served, monitored, and maintained after launch.
• Prior experience as the senior-most or near-senior-most engineer on a production ML or data platform, with responsibility for architectural decisions others had to live with.
• A track record of setting technical standards that outlasted your own involvement, and of leading through influence without formal authority.
• Strong hands-on Python and SQL , and the software engineering fundamentals that keep ML systems maintainable at scale - version control, automated testing, code review, modular design - plus the discipline to enforce them across teams.
• Expert hands‑on proficiency with Databricks (Spark, Lakeflow Declarative Pipelines, Delta Lake, Lakebase/Postgres, Unity Catalog, Model Serving) and the Azure data ecosystem. Deep experience on a comparable lakehouse platform will be considered, with the expectation of becoming effective on Databricks quickly.
• Experience with MLflow or an equivalent for experiment tracking, model registry, tracing, and reproducibility.
• At least one year building agentic systems that reached production use, or equiv