Remote Model Serving Engineer – Scalable ML Inference

Bright Vision Technologies; Schaumburg (IL); Remote; USD 74,000 - 98,000; Full time

Job summary

Bright Vision Technologies is seeking a Model Serving Engineer to design, build, and operate high-performance inference platforms for serving large ML models in production. The role emphasizes the systems engineering side of AI deployment, including request routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across diverse model workloads.

The ideal candidate brings strong distributed systems and performance engineering expertise and has shipped serving systems

Qualifications

• Bachelor’s or Master’s degree in Computer Science or a related field.
• 7+ years of experience in distributed systems, infrastructure, or ML platform engineering.
• Strong proficiency in Python and a systems language such as Go, Rust, or C++.
• Experience operating high-throughput, low-latency services in production.
• Hands-on experience with LLM or large model inference frameworks such as vLLM or TensorRT-LLM.
• Strong understanding of GPU architecture, memory hierarchies, and accelerator utilization.
• Familiarity with Kubernetes, autoscaling, and modern cloud platforms.
• Experience with observability stacks including metrics, tracing, and structured logging.
• Solid grounding in performance engineering and capacity planning.
• Strong communication and incident response skills.

Responsibilities

• Design, build, and operate high-performance inference platforms for serving large ML models.
• Focus on distributed systems, performance engineering, and end-to-end observability.
• Balance latency, throughput, and cost while maintaining quality in production.

Skills

Python

Go/Rust/C++

Distributed systems

Performance engineering

GPU architecture

Kubernetes

Observability

Incident response

LLM frameworks

Autoscaling

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

TensorRT-LLM

vLLM

Kubernetes

Job description

Bright Vision Technologies is seeking a Model Serving Engineer to design, build, and operate high-performance inference platforms for serving large ML models in production. The role emphasizes the systems engineering side of AI deployment, including request routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across diverse model workloads.

The ideal candidate brings strong distributed systems and performance engineering expertise and has shipped serving systems