Senior Machine Learning Engineer

Kavaliro

Neptune Beach (FL)

Remote

USD 96,000 - 103,000

Full time

5 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Kavaliro is seeking an experienced Senior Machine Learning Engineer with deep expertise in learning-to-rank systems for a large-scale digital marketplace. The role emphasizes ranking algorithms, unbiased evaluation, and experimentation design, with remote work options in the United States.

You will mentor engineers while shaping the next generation of ranking models and feature-rich, session-aware recommendations.

Qualifications

  • Authorization to work in the United States.
  • Hands-on experience building production learning-to-rank systems for search, recommendations, advertising, or marketplace apps.
  • Strong understanding of ranking methodologies: pointwise, pairwise, listwise.
  • Expertise with ranking metrics: NDCG, MRR, MAP and related eval techniques.
  • Experience addressing position bias, selection bias, and counterfactual learning.
  • Strong online experimentation, A/B testing, stats, and performance measurement.
  • Experience with multi-objective optimization and funnel-based ranking.
  • Advanced Python and modern ML frameworks proficiency.
  • Ability to communicate findings and influence decisions with data-driven insights.
  • Experience with large-scale marketplace/e-commerce/travel/services platforms.
  • Experience modeling delayed feedback and time-dependent conversions.
  • Experience with session-aware/context-aware ranking systems.
  • Hands-on cloud-based ML platforms and large-scale data pipelines.
  • Experience in highly regulated industries.
  • Prior consulting or client-facing experience including mentoring.

Responsibilities

  • Investigate discrepancies between offline model performance and production results.
  • Improve evaluation frameworks so offline metrics better predict real-world impact.
  • Design stronger experimentation methodologies, incl. A/B testing and stats.
  • Apply counterfactual and unbiased learning-to-rank techniques to address bias.
  • Develop multi-objective ranking approaches balancing competing metrics.
  • Incoporate delayed feedback and incomplete conversion signals into training.
  • Model session-level user behavior with richer contextual features.
  • Train, optimize, and evaluate ranking models on large-scale datasets.
  • Define rollout and validation strategies for production deployment.
  • Collaborate with stakeholders and mentor internal engineers.

Skills

Learning-to-rank
Ranking methodologies (pointwise, pair
NDCG/MRR/MAP
Online experimentation
Python
Cloud ML platforms
Mentoring/knowledge transfer
Regulated industries
Large-scale datasets
Data-driven communication

Tools

Python-based ML stack
Deep learning frameworks
Gradient-boosted ranking

Job description

Senior Machine Learning Engineer - Learning-to-Rank Location: United States (Remote)

About the Role

We are seeking an experienced Machine Learning Engineer with deep expertise in learning-to-rank systems to support a large-scale digital marketplace. This platform connects consumers with service providers and relies heavily on search and recommendation algorithms to drive successful matches. The organization recently transitioned from a rules-based ranking approach to a neural network powered ranking model. While initial offline evaluation showed promising improvements, production performance revealed more nuanced outcomes across user journeys, conversion stages, and audience segments. This role focuses on understanding those discrepancies and helping design, build, and launch the next generation ranking system. The work is centered on ranking algorithms, experimentation, evaluation methodologies, and machine learning modeling rather than infrastructure or systems architecture.

Key Challenges
  • Offline-to-online alignment: Traditional offline ranking metrics do not consistently predict real-world business outcomes.
  • Position and selection bias: Training data is generated from previously displayed results, creating biased feedback loops.
  • Multi-objective optimization: Multiple user and business objectives are currently combined into a single ranking target, masking important tradeoffs.
  • Session-aware behavior: Users frequently perform multiple searches before making a decision.
  • Delayed outcomes: Many conversions occur well after the initial search event, resulting in incomplete training labels.
  • Feature utilization: Contextual signals such as device type, channel, and behavioral patterns are not fully leveraged.
What You'll Do
  • Investigate discrepancies between offline model performance and production results.
  • Improve evaluation frameworks so offline metrics better predict real-world impact.
  • Design stronger experimentation methodologies, including A/B testing strategy and statistical analysis.
  • Apply counterfactual and unbiased learning-to-rank techniques to address position and selection bias.
  • Develop multi-objective ranking approaches that explicitly balance competing success metrics.
  • Incorporate delayed feedback and incomplete conversion signals into model training.
  • Model session-level user behavior and introduce richer contextual features.
  • Train, optimize, and evaluate ranking models on large-scale datasets.
  • Define rollout and validation strategies for production deployment.
  • Collaborate closely with technical stakeholders and mentor internal engineering team members.
Technical Environment
  • Large-scale training datasets containing tens of millions of examples
  • Cloud-native machine learning infrastructure
  • Python-based machine learning stack
  • Deep learning and gradient-boosted ranking frameworks
  • Distributed data storage and model training environments
What You Bring
  • Required Authorization to work in the United States.
  • Hands-on experience building and deploying production learning-to-rank systems for search, recommendations, advertising, or marketplace applications.
  • Strong understanding of ranking methodologies, including pointwise, pairwise, and listwise approaches.
  • Expertise with ranking metrics such as NDCG, MRR, MAP, and related evaluation techniques.
  • Experience addressing position bias, selection bias, and counterfactual learning problems.
  • Strong background in online experimentation, A/B testing, statistical analysis, and performance measurement.
  • Experience with multi-objective optimization and funnel-based ranking systems.
  • Advanced proficiency in Python and modern machine learning frameworks.
  • Ability to communicate technical findings and influence business decisions through data-driven recommendations.
  • Preferred Experience working with large-scale marketplace, e-commerce, services, travel, healthcare, or recommendation platforms.
  • Experience modeling delayed feedback, censored labels, or time-dependent conversion outcomes.
  • Experience with session-aware or context-aware ranking systems.
  • Hands-on experience with cloud-based machine learning platforms and large-scale data pipelines.
  • Experience in highly regulated industries.
  • Prior consulting or client-facing experience, including mentoring and knowledge transfer responsibilities.
Why This Opportunity

This engagement offers the chance to solve challenging real-world ranking problems at scale, directly influencing how users discover and connect with relevant service providers. You'll work on advanced machine learning topics including unbiased learning-to-rank, experimentation design, multi-objective optimization, and delayed-feedback modeling while helping shape the evolution of a mission-critical recommendation and search platform.

Pay

Pay: $70 - $75/Hourly

Kavaliro provides Equal Employment Opportunities to all employees and applicants.

All qualified applicants will receive consideration for employment without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws.

Kavaliro is committed to the full inclusion of all qualified individuals.

In keeping with our commitment, Kavaliro will take the steps to assure that people with disabilities are provided reasonable accommodations.

Accordingly, if reasonable accommodation is required to fully participate in the job application or interview process, to perform the essential functions of the position, and/or to receive all other benefits and privileges of employment, please respond to this posting to connect with a company representative.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI/ML Engineer
Senior AI/ML Engineer

Kavaliro • Florida

On-site
USD 110,000 - 130,000
Senior AI/ML Engineer — Remote, MLOps & AI Systems
Senior AI/ML Engineer — Remote, MLOps & AI Systems

Kavaliro • Florida

Remote
USD 140,000 - 200,000
Senior Machine Learning Engineer - Ranking
Senior Machine Learning Engineer - Ranking

Quinstreet • United States

Remote
USD 140,000 - 170,000
Performance bonus
Equity in RSUs
Health insurance
+2
Remote ML Engineer - Ranking & Retrieval for Search
Remote ML Engineer - Ranking & Retrieval for Search

ClickUp • United States

Hybrid
USD 120,000 - 180,000
Machine Learning Engineer, Ranking & Retrieval (Remote)
Machine Learning Engineer, Ranking & Retrieval (Remote)

MAP SSG • United States

Remote
USD 185,000 - 280,000
Equity
Discretionary bonus
Remote work
Senior Software Engineer, 1
Senior Software Engineer, 1

Dotdash Meredith • Chicago (IL)

Hybrid
USD 125,000 - 150,000
Senior Software Engineer, 1
Senior Software Engineer, 1

Dotdash Meredith • Los Angeles (CA)

Hybrid
USD 125,000 - 150,000
Senior Software Engineer, 1
Senior Software Engineer, 1

Dotdash Meredith • West Hollywood (CA)

Hybrid
USD 125,000 - 150,000
Senior Software Engineer, 1
Senior Software Engineer, 1

Dotdash Meredith • Des Moines (IA)

Hybrid
USD 125,000 - 150,000
Staff Machine Learning Engineer - Retention
Staff Machine Learning Engineer - Retention

Taskrabbit • San Francisco (CA)

Hybrid
USD 175,000 - 230,000
Employer-paid health insurance
401k match
Generous time off
+2