Remote Model Serving Engineer – ML Inference at Scale

Bright Vision Technologies; Aurora (IL); Remote; USD 74,000 - 98,000; Full time

Job summary

Bright Vision Technologies is hiring a Model Serving Engineer to design, build, and operate high-performance inference platforms for production ML workloads. The role emphasizes systems engineering, request routing, batching, caching, autoscaling, and GPU utilization with end-to-end observability across diverse model workloads.

The ideal candidate has strong distributed systems and performance engineering experience, has shipped serving systems at scale, and can balance latency, throughput,

Qualifications

• Bachelor's or Master’s degree in Computer Science or a related field.
• Six or more years of experience in distributed systems, infrastructure, or ML platform engineering.
• Strong proficiency in Python and a systems language such as Go, Rust, or C++.

Responsibilities

• Design, build, and operate high-performance inference platforms for production ML workloads.
• Focus on systems engineering: request routing, batching, caching, autoscaling, and GPU utilization.
• Ensure end-to-end observability across diverse model workloads with metrics, tracing, and logging.

Skills

Distributed systems

Python

Go/Rust/C++

Kubernetes

Performance engineering

Observability

Low-latency systems

Incident response

GPU runtimes

Education

Bachelor's or Master's in CS

Tools

vLLM

TensorRT-LLM

Cloud platforms

Job description

Bright Vision Technologies is hiring a Model Serving Engineer to design, build, and operate high-performance inference platforms for production ML workloads. The role emphasizes systems engineering, request routing, batching, caching, autoscaling, and GPU utilization with end-to-end observability across diverse model workloads.

The ideal candidate has strong distributed systems and performance engineering experience, has shipped serving systems at scale, and can balance latency, throughput,