Machine Learning Engineer 5 - Decisioning & Optimization

Netflix, Inc.; New York (NY); On-site; USD 466,000 - 750,000; Full time

Benefits offered by this job

Health plans

401(k) retirement plan

Flexible time off

Paid leave of absence

Job summary

Netflix, Inc. is looking for a skilled engineer for their Decisioning & Optimization team. This role involves building and maintaining ML model serving infrastructure for real-time ad decisioning, ensuring system reliability, and optimizing performance for numerous concurrent models.

The ideal candidate will have extensive software engineering experience, particularly in machine learning infrastructure, and should be proficient in Java, Python, or Scala. This position offers a competitive salary ranging from $466,000 to $750,000 and includes various benefits to support overall well-being.

Qualifications

• 7+ years of software engineering experience, with 3+ years in ML infrastructure.
• Experience with real-time model serving systems with sub‑20ms latency.
• Proficiency in Java, Python, or Scala, focusing on performance optimization.

Responsibilities

• Build and operate end‑to‑end ML model serving infrastructure for real‑time ad decisioning.
• Design and optimize feature serving paths for latency and online/offline consistency.
• Drive operational excellence for ML systems: reliability and incident response.

Skills

Software engineering

Machine learning infrastructure

Java, Python, or Scala

Model serving systems

Real‑time decisioning

Feature engineering

Model monitoring

Multi-threading

Job description

The Decisioning & Optimization engineering team owns the systems that determine which ad wins every impression, at what price, and how campaign budgets deliver across all inventory surfaces. Our work spans three platform areas: ML infrastructure for model serving, auction/ranking/scoring, and budget/pacing/bidding.

What You'll Do

• Build and operate end‑to‑end ML model serving infrastructure for real‑time ad decisioning: model publishing, packaging, validation, and deployment into the serving stack with zero‑downtime hot‑swap.
• Scale the inference path to support dozens of concurrent models on every ad request at 1M+ QPS with strict latency budgets, including batching strategies, CPU/GPU allocation, model versioning, and fallback tiers.
• Design and optimize the feature serving path: feature hydration from Chronon, Signal Service, and real‑time streams with sub‑10ms P99 fetch latency and online/offline consistency.
• Productionize scoring and ranking models for multi‑stage ad selection (retrieval, early ranking, full scoring) and integrate model outputs into auction.
• Build model performance monitoring in production: inference latency, prediction distribution shifts, feature drift detection, score calibration, and regression detection before revenue impact.
• Partner closely with Data Science & Platform teams.
• Build simulation infrastructure to replay production traffic against candidate models offline, enabling validation of marketplace changes before live rollout.
• Drive operational excellence for ML systems: reliability, observability, capacity planning, incident response, and scaling for live events with 35M+ concurrent viewers.

Skills & Experience

• 7+ years of software engineering experience; 3+ years focused on ML infrastructure, model serving, or ML platform work in an ads or real‑time decisioning context.
• Built and operated real‑time model serving systems at high QPS with sub‑20ms latency: online inference, feature stores, model registries, model hot‑swap, canary and shadow rollout.
• Proficiency in Java, Python, or Scala with a solid understanding of multi‑threading, memory management, and performance optimization for latency‑critical paths.
• Hands‑on experience with ML serving frameworks: serialization, runtime optimization, and deployment constraints.
• Experience with feature engineering pipelines for real‑time systems: online/offline consistency, hydration strategies, caching, and freshness tradeoffs.
• Strong understanding of model monitoring in production: drift detection, prediction distribution analysis, calibration, and latency profiling.
• Comfortable working at the boundary between ML research and production engineering: can take a model artifact and turn it into a production‑ready service that meets SLA.
• Demonstrated ability to operate in an environment that requires both big‑tech scale and startup speed.
• Nice to have: Ads domain experience (ranking models, bid scoring, reserve pricing, yield optimization, dynamic allocation across guaranteed and non‑guaranteed inventory).
• Nice to have: Experience with auction mechanics (multi‑stage ranking, bid shading, bid prediction, marketplace competition dynamics).
• Nice to have: Built or improved budget pacing and delivery control systems.
• Nice to have: Built simulation or counterfactual testing platforms for marketplace or auction systems.
• Nice to have: Experience with A/B testing infrastructure for model rollouts (online experiments, holdout groups, interference‑aware evaluation in marketplace settings).
• Nice to have: Familiar with CTV constraints (server‑side ad insertion, live event ad serving at scale, burst traffic patterns).
• Nice to have: Experience with JVM ecosystem.

Compensation & Benefits

• Annual salary range: $466,000.00 – $750,000.00.
• Benefits include health plans, mental health support, 401(k) retirement plan with employer match, stock option program, disability programs, health savings and flexible spending accounts, family‑forming benefits, life and serious injury benefits, paid leave of absence, and flexible time off.

We are an equal‑opportunity employer and celebrate diversity, recognizing that diversity builds stronger teams. We approach diversity and inclusion seriously and thoughtfully. We do not discriminate on the basis of race, religion, color, ancestry, national origin, caste, sex, sexual orientation, gender, gender identity or expression, age, disability, medical condition, pregnancy, genetic makeup, marital status, or military service.