Remote Model Serving Engineer — Scalable ML Inference

Bright Vision Technologies; United States; Remote; USD 74,000 - 98,000; Full time

Job summary

Bright Vision Technologies is seeking a Model Serving Engineer to design, build, and operate high-performance inference platforms for large machine learning models in production. The role emphasizes the systems engineering side of AI deployment, including routing, batching, caching, autoscaling, GPU utilization, and observability.

The ideal candidate brings 7+ years in distributed systems or ML platform engineering, strong Python and Go/Rust/C++, and hands-on experience with LLM frameworks such

Qualifications

• Degree in CS or related field.
• 7+ years in distributed systems or ML platform engineering.
• Proficiency in Python and a systems language (Go/Rust/C++).
• Experience running high-throughput, low-latency services in production.
• Hands-on with LLM inference frameworks such as vLLM or TensorRT-LLM.
• Strong knowledge of GPUs, memory hierarchies, and accelerators.
• Familiarity with Kubernetes and cloud platforms.
• Experience with observability stacks (metrics, tracing, logging).
• Strong communication and incident response skills.

Responsibilities

• Design, build, and operate inference platforms for ML models.
• Optimize request routing, batching, caching, autoscaling, GPU utilization.
• Ensure end-to-end observability across model workloads.
• Collaborate to balance latency, throughput, and cost.
• Maintain production reliability and respond to incidents.

Skills

Distributed systems

Python

Go/Rust/C++

High-throughput/low-latency

LLM inference frameworks

GPU architectures

Kubernetes & cloud

Observability

Performance engineering

Incident response

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

Kubernetes

Cloud platforms

LLM inference frameworks (vLLM/TensorRT-LLM)

Auto-scaling

Job description

Bright Vision Technologies is seeking a Model Serving Engineer to design, build, and operate high-performance inference platforms for large machine learning models in production. The role emphasizes the systems engineering side of AI deployment, including routing, batching, caching, autoscaling, GPU utilization, and observability.

The ideal candidate brings 7+ years in distributed systems or ML platform engineering, strong Python and Go/Rust/C++, and hands-on experience with LLM frameworks such