Senior ML Infrastructure Engineer - Online Inference

Unity Enterprise; Northern (KY); Hybrid; USD 210,000 - 273,000; Full time

Benefits offered by this job

Comprehensive health insurance

Employee stock ownership

Generous vacation and personal days

Commuting subsidy

Job summary

Unity Enterprise is seeking a Senior ML Engineer to design and evolve an online inference platform for production ML models. You will build reliable infrastructure for serving models, optimize inference performance, and enable safe experimentation across high-traffic systems.

Collaboration with ML engineers and platform teams will be crucial to deploy, monitor, and iterate models at scale. You will lead architectural improvements, design canary and A/B rollout strategies, and drive

Qualifications

• Experience building and operating production-grade online ML inference systems such as Triton, TorchServe, Ray Serve or TensorFlow Serving.
• Strong background in distributed systems, Kubernetes, autoscaling, reliability, and observability.
• Proficient in Python and production ML workflows, including packaging, validation and lifecycle management.

Responsibilities

• Design and operate large-scale online inference infrastructure for production ML models with low latency and high reliability.
• Optimize model performance with techniques like quantization, CUDA/kernel optimization, and dynamic batching.
• Improve observability of ML systems and support rapid, safe model iteration across teams.
• Lead architectural improvements to make the platform robust, scalable, and cost-efficient.

Skills

Production ML infra

Distributed systems

Python programming

Model deployment workflows

Canary testing & A/B

Tools

NVIDIA Triton Inference Server

TorchServe

Kubernetes

Flyte / Airflow

Ray

PyTorch

Job description

Unity Enterprise is seeking a Senior ML Engineer to design and evolve an online inference platform for production ML models. You will build reliable infrastructure for serving models, optimize inference performance, and enable safe experimentation across high-traffic systems.

Collaboration with ML engineers and platform teams will be crucial to deploy, monitor, and iterate models at scale. You will lead architectural improvements, design canary and A/B rollout strategies, and drive