Senior ML Infrastructure Engineer — Remote

Bright Vision Technologies; Palo Alto (CA); On-site; USD 105,000 - 143,000; Full time

Job summary

Bright Vision Technologies is seeking a Machine Learning Infrastructure Engineer to design, build, and operate high-performance inference platforms for serving large ML models in production. The role emphasizes systems engineering for AI deployment, including request routing, batching, caching, autoscaling, GPU utilization, and observability across diverse model workloads.

The ideal candidate brings strong distributed systems expertise, has shipped serving systems at scale, and understands

Qualifications

• Bachelor's or Master's degree in Computer Science or a related field.
• 6+ years of experience in distributed systems, infrastructure, or ML platform engineering.
• Strong proficiency in Python and a systems language (Go, Rust, or C++).
• Deep experience operating high-throughput, low-latency services in production.
• Hands-on experience with LLM or large model inference frameworks (vLLM or TensorRT-LLM).
• Strong understanding of GPU architecture, memory hierarchies, and accelerator utilization.
• Familiarity with Kubernetes, autoscaling, and modern cloud platforms.
• Experience with observability stacks including metrics, tracing, and structured logging.
• Solid grounding in performance engineering and capacity planning.
• Strong communication and incident response skills.

Responsibilities

• Design, build, and operate high-performance inference platforms for serving large ML models in production.
• Focus on systems engineering side of AI deployment: routing, batching, caching, autoscaling, and GPU utilization.
• Establish end-to-end observability across diverse model workloads and ensure reliability at scale.
• Collaborate with teams to balance latency, throughput, cost, and quality in ML serving.

Skills

Python

Go

Rust

C++

Distributed systems

ML platform engineering

Kubernetes

observability

performance engineering

capacity planning

incident response

Education

Bachelor's or Master's in Computer Science or related field

Tools

Kubernetes

vLLM

TensorRT-LLM

Job description

Bright Vision Technologies is seeking a Machine Learning Infrastructure Engineer to design, build, and operate high-performance inference platforms for serving large ML models in production. The role emphasizes systems engineering for AI deployment, including request routing, batching, caching, autoscaling, GPU utilization, and observability across diverse model workloads.

The ideal candidate brings strong distributed systems expertise, has shipped serving systems at scale, and understands