Remote ML Platform Engineer — Scalable Inference Systems

Bright Vision Technologies; Nashua (NH); On-site; USD 100,000 - 160,000; Full time

Job summary

Bright Vision Technologies is seeking an ML Platform Engineer to design, build, and operate high-performance inference platforms for large AI models. This remote role focuses on distributed systems, request routing, batching, autoscaling, and observability across model workloads.

The ideal candidate has 10+ years in ML infrastructure, strong Python and systems language skills, and experience with LLM frameworks.

Qualifications

• 10+ years in distributed systems, infra, or ML platform work.
• Bachelor’s degree or higher in CS or related field required.
• Experience running high-throughput, low-latency production services is essential.

Responsibilities

• Design and operate model serving platforms for diverse workloads.
• Optimize inference performance with batching, caching, and efficient scheduling.
• Implement multi-tenant routing, rate limiting, and QoS across model endpoints.
• Build autoscaling and capacity management to balance latency, throughput and cost.
• Tune GPU utilization, memory for LLM serving workloads.
• Drive end-to-end observability and incident response across services.

Skills

Distributed systems

Python

Go/Rust/C++

LLM inference frameworks

Kubernetes

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

vcLLM

TensorRT-LLM

Job description

Bright Vision Technologies is seeking an ML Platform Engineer to design, build, and operate high-performance inference platforms for large AI models. This remote role focuses on distributed systems, request routing, batching, autoscaling, and observability across model workloads.

The ideal candidate has 10+ years in ML infrastructure, strong Python and systems language skills, and experience with LLM frameworks.