Senior ML Infra Engineer (Scala) & Orchestration Platform

Etsy, Inc.; New York (NY); Hybrid; USD 164,000 - 246,000; Full time

Benefits offered by this job

Equity package

Annual bonus

Competitive benefits

Job summary

Etsy, Inc. is seeking a Senior Machine Learning Infrastructure Engineer to build and run Etsy’s ML orchestration platform. This high-scale system handles requests across search, recommendations, ads, trust and safety, and generative AI workloads, ensuring low latency and high availability for buyers and sellers.

You’ll join ML Enablement, a platform team focused on making production ML safe, fast, and self-service, with design reviews, mentorship, and cross-team collaboration.

Qualifications

• 5+ years of software engineering experience building and operating distributed, low-latency production services.
• Experience with Scala or another functional programming language and comfort with functional patterns.
• Hands-on experience operating services on Kubernetes, including deployments, scaling, resource management, and debugging in production.
• Experience designing service APIs and request-routing or orchestration systems (e.g., gRPC, queues).
• Strong operational skills: metrics, tracing, alerting, on-call ownership, and performance profiling.

Responsibilities

• Design, build, and operate the services that route, orchestrate, and execute online ML inference requests at scale, with a focus on latency, throughput, reliability, and observability.
• Build orchestration capabilities such as multi-model request graphs, feature retrieval, fan-out and fan-in, caching, batching, fallbacks, timeouts, and graceful degradation.
• Enable safe model rollout and evaluation: shadow traffic, canaries, A/B and experiment integration, and online and offline evaluation workflows.
• Support both predictive models and generative workloads, including the orchestration patterns they need, such as streaming, long-running requests, and GPU-backed serving.
• Build self-service interfaces (APIs, SDKs, configuration, tooling) so scientists and engineers can onboard and ship models without platform expertise.
• Define and uphold SLOs and capacity plans. Lead monitoring, alerting, incident response, and postmortems for a tier-one system.
• Partner with users to gather feedback, drive adoption of paved paths, and plan migrations away from legacy integrations.
• Contribute to technical excellence through design reviews, code reviews, and mentorship.

Skills

Distributed systems

Scala

Kubernetes

APIs & routing

Observability

Tools

gRPC

HTTP4S / Scala libraries

Job description

Etsy, Inc. is seeking a Senior Machine Learning Infrastructure Engineer to build and run Etsy’s ML orchestration platform. This high-scale system handles requests across search, recommendations, ads, trust and safety, and generative AI workloads, ensuring low latency and high availability for buyers and sellers.

You’ll join ML Enablement, a platform team focused on making production ML safe, fast, and self-service, with design reviews, mentorship, and cross-team collaboration.