Senior Machine Learning Engineer, ML Infrastructure- Online

Unity Technologies SF

Olympia (WA)

On-site

USD 166,000 - 273,000

Full time

5 days ago

Be an early applicant

Application generator

Get a reply from this employer — a resume and cover letter tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Comprehensive health insurance

Commute subsidy

Employee stock ownership

Generous vacation and personal days

Mental Health programs

Volunteering and donation matching

Job summary

Unity Technologies SF is seeking a Senior Machine Learning Engineer to design and evolve Unity Vector’s online inference platform. The role focuses on production infrastructure for low-latency inference, with emphasis on optimization, observability, and safe experimentation.

You will design and operate large-scale online inference infrastructure, build distributed training support, and drive end-to-end serving lifecycle management.

Qualifications

Experience building production-grade online ML inference systems.
Experience with model serving frameworks and distributed serving platforms.
Strong background in distributed systems and production observability.
Proficiency in Python for high-scale services.
Experience with PyTorch and modern deployment workflows.

Responsibilities

Design and operate large-scale online inference infrastructure for production ML models.
Build infrastructure for distributed training workflows.
Integrate ML pipelines with workflow orchestration systems.
Optimize model performance and resource utilization.
Enhance ML system observability and reliability.
Collaborate to accelerate model iteration while maintaining safety.
Improve deployment automation and reproducibility of serving workflows.
Lead architectural improvements for robustness and cost efficiency.

Skills

Python programming

Distributed systems

Kubernetes

ML inference

Low-latency optimization

Observability

Canary / A/B testing

Canary testing

Production ML systems

Tools

PyTorch

NVIDIA Triton Inference Server

TorchServe

Ray Serve

TensorFlow Serving

Kubernetes

GKE

Ray

Flyte

Airflow

Ray Data

Ray Train

Python

Job description

In this Senior Machine Learning Engineer role, you will help design and evolve Unity Vector’s online model inference platform. The focus is on production infrastructure for low-latency, high-reliability inference, with strong emphasis on optimization, observability, and safe experimentation.

Responsibilities

Design and operate large-scale online inference infrastructure that serves production ML models with low latency and high reliability, including solutions such as PyTorch, Triton Inference Server, Kubernetes, GKE, Ray, or comparable distributed serving frameworks.
Build infrastructure that supports distributed training workflows using tools such as Pytorch, Ray Data, and Ray Train.
Integrate ML pipelines with workflow orchestration systems (such as Flyte, Airflow, or similar) to support dependable multi-stage training workflows.
Optimize model performance through techniques including model compilation, GPU and CPU utilization improvements, request scheduling, kernel fusion, and runtime-level tuning.
Enhance ML system observability by monitoring latency, throughput, error rates, cost, saturation, and model health.
Collaborate with ML engineers to accelerate model iteration while maintaining production safety, scalability, and cost efficiency.
Improve reliability and reproducibility of model serving workflows, including model packaging, artifact validation, compatibility testing, and deployment automation.
Lead architectural improvements to increase robustness, usability, scalability, and cost efficiency for the online ML platform.

Requirements

Experience building and operating production-grade online ML inference systems, such as NVIDIA Triton Inference Server, TorchServe, Ray Serve, TensorFlow Serving, or similar platforms.
Experience with model serving frameworks including NVIDIA Triton Inference Server, TorchServe, Ray Serve, TensorFlow Serving, or similar systems.
Experience optimizing inference workloads using dynamic batching, model compilation, quantization, GPU acceleration, GPU kernel optimization, caching, or runtime tuning.
Strong background in distributed systems, Kubernetes, autoscaling, service reliability, and production observability.
Strong programming skills in Python, including practical experience in production ML systems and high-scale services.
Experience with PyTorch and modern model deployment workflows, covering model packaging, validation, and end-to-end serving lifecycle management.
Experience designing infrastructure for safe rollout approaches such as canary testing, A/B experimentation, and automated rollback.
Strong systems thinking to balance latency, throughput, reliability, scalability, and cost tradeoffs in online environments.
Demonstrated ability to lead technical direction and influence architectural decisions across teams without formal authority.

Technologies