Lead Machine Learning Engineer

Harnham; New York (NY); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Equity participation via RSUs

Hybrid NYC

Job summary

Harnham seeks a Lead Machine Learning Engineer to join a rapid growth ML engineering team in NYC, offering hybrid work. You will design scalable ML infrastructure, build backend services, and collaborate with Data Scientists to productionize models for real-time recommendations.

The role requires 7+ years in software or ML engineering, strong backend experience with distributed systems, and hands-on skills in Scala, AWS, Kubernetes, and CI/CD.

Qualifications

• 7+ years of software or ML engineering experience.
• Backend engineering experience building distributed systems at scale.
• Scala-based microservices and production-grade backend applications.
• Deep knowledge of AWS cloud services, including ML infrastructure.
• Hands-on with Kubernetes, Docker, Terraform, and modern CI/CD practices.

Responsibilities

• Design, build, and maintain scalable infrastructure for ML training, deployment, and inference.
• Develop and optimize backend services and cloud-native applications powering real-time ML systems.
• Own ML platform capabilities across cloud infrastructure, model serving, monitoring, and tooling.
• Collaborate with Data Scientists to productionize models for real-time recommendations.
• Improve CI/CD pipelines, IaC, observability, reliability, and scalability.
• Participate in on-call, incident response, and support for critical production services.

Skills

Backend engineering

Distributed systems

Scala

AWS

Kubernetes

Docker

CI/CD

Observability

Tools

Datadog

SageMaker

Python

Job description

THE COMPANY

We are partnering with a leading consumer technology and financial services organization that operates at global scale and serves hundreds of millions of users. The business leverages advanced data, machine learning, and real-time decisioning systems to deliver highly personalized customer experiences across a broad portfolio of digital products.

This is an exciting opportunity to join a rapidly expanding machine learning engineering team at a pivotal stage of growth. The organization is investing heavily in recommendation systems, real-time personalization, machine learning platforms, and next-generation AI capabilities, offering engineers the opportunity to work with large-scale distributed systems and production-grade ML infrastructure.

RESPONSIBILITIES

• Design, build, and maintain scalable infrastructure supporting machine learning training, deployment, and inference workloads.
• Develop and optimize backend services, microservices, and cloud-native applications that power real-time machine learning systems.
• Own and enhance ML platform capabilities across cloud infrastructure, model serving, monitoring, and operational tooling.
• Partner closely with Data Scientists to productionize machine learning models and support real-time recommendation and personalization use cases.
• Improve CI/CD pipelines, infrastructure-as-code frameworks, observability, reliability, and system scalability.
• Participate in operational ownership, incident response, and support for critical production services.

SKILLS AND EXPERIENCE

Must-Have

• 7+ years of software engineering or machine learning engineering experience.
• Strong backend engineering expertise with experience building distributed systems at scale.
• Proven experience developing Scala-based microservices and production-grade backend applications.
• Deep knowledge of AWS cloud services, including machine learning infrastructure and managed platforms.
• Hands-on experience with Kubernetes, Docker, Terraform, and modern CI/CD practices.
• Track record of owning production systems, reliability, monitoring, and operational excellence.

Nice-to-Have

• Experience with machine learning infrastructure, MLOps, or model-serving platforms.
• Knowledge of recommendation systems, personalization engines, or CTR optimization.
• Experience with Datadog observability and monitoring.
• Background in adtech, fintech, e-commerce, or other high-scale consumer platforms.
• Exposure to real-time machine learning applications and online inference systems.

BENEFITS

• Competitive base salary and annual bonus
• Equity participation through RSUs
• Opportunity to work on cutting-edge AI and machine learning initiatives
• Significant career growth and technical leadership opportunities
• Exposure to large-scale, real-time production systems

KEY TERMS

Lead Machine Learning Engineer | Machine Learning Engineering | ML Infrastructure | Scala | Python | AWS | SageMaker | Kubernetes | Docker | Terraform | CI/CD | Distributed Systems | Recommendation Systems | Real-Time Systems | MLOps | Backend Engineering | Cloud Infrastructure | Platform Engineering | Datadog | Fintech | Personalization | ML Platform | Software Engineering | Hybrid NYC | Technical Leadership