Remote Model Serving Engineer for Scalable AI Inference

Bright Vision Technologies; United States; Remote; USD 74,000 - 98,000; Full time

Job summary

Bright Vision Technologies is seeking a Model Serving Engineer to design, build, and operate high-performance inference platforms for serving large machine learning models in production.

The role emphasizes distributed systems, performance engineering, and observability, with a strong focus on latency, throughput, and cost trade-offs across varied model workloads. This is a 100% remote, full-time position within the U.S.

Qualifications

• Bachelor’s or Master’s degree in Computer Science or a related field.
• Six or more years of experience in distributed systems, infrastructure, or ML platform engineering.
• Strong proficiency in Python and a systems language such as Go, Rust, or C++.
• Deep experience operating high-throughput, low-latency services in production.
• Hands-on experience with LLM or large model inference frameworks such as vLLM or TensorRT-LLM.
• Strong understanding of GPU architecture, memory hierarchies, and accelerator utilization.
• Familiarity with Kubernetes, autoscaling, and modern cloud platforms.
• Experience with observability stacks including metrics, tracing, and structured logging.
• Solid grounding in performance engineering and capacity planning.
• Strong communication and incident response skills.

Responsibilities

• Design, build, and operate high-performance inference platforms for production ML models.
• Focus on systems engineering of AI deployment including routing, batching, caching, autoscaling, GPU utilization.
• Ensure end-to-end observability across diverse model workloads and manage latency vs. cost trade-offs.

Skills

Distributed systems

Python

Go/Rust/C++

High-throughput/low-latency

LLM inference frameworks (vLLM, Tensor

Education

Bachelor’s or Master’s in CS or related

Tools

Kubernetes

Observability stacks

Cloud platforms

GPU/accelerator utilization

Job description

Bright Vision Technologies is seeking a Model Serving Engineer to design, build, and operate high-performance inference platforms for serving large machine learning models in production.

The role emphasizes distributed systems, performance engineering, and observability, with a strong focus on latency, throughput, and cost trade-offs across varied model workloads. This is a 100% remote, full-time position within the U.S.