Remote ML Infrastructure Engineer: Scale AI Inference

JobCubby; Redwood City (CA); Hybrid; USD 105,000 - 143,000; Full time

Job summary

Bright Vision Technologies is seeking a Machine Learning Infrastructure Engineer to design, build, and operate high-performance inference platforms for production ML models. You will focus on the systems engineering of AI deployment, including routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across diverse workloads.

The ideal candidate has deep distributed systems and performance engineering expertise, has shipped serving systems at scale, and understands

Qualifications

• Bachelor’s or Master’s degree in Computer Science or a related field.
• Six+ years of experience in distributed systems or ML platform engineering.
• Strong proficiency in Python and a systems language (Go, Rust, or C++).
• Deep experience operating high-throughput, low-latency services in production.
• Experience with LLM inference frameworks such as vLLM or TensorRT-LLM.

Responsibilities

• Design, build, and operate high-performance inference platforms for production ML models.
• Focus on the systems engineering side of AI deployment, including routing, batching, caching, autoscaling, and GPU utilization.
• Provide end-to-end observability across diverse model workloads and optimize latency, throughput, and cost.

Skills

Python

Go

Rust

C++

Distributed systems

Performance engineering

Education

Bachelor’s or Master’s degree in Computer Science

Tools

Kubernetes

TensorRT-LLM

vLLM

Job description

Bright Vision Technologies is seeking a Machine Learning Infrastructure Engineer to design, build, and operate high-performance inference platforms for production ML models. You will focus on the systems engineering of AI deployment, including routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across diverse workloads.

The ideal candidate has deep distributed systems and performance engineering expertise, has shipped serving systems at scale, and understands