Remote MLOps Engineer — Scalable AI Inference Platform

Bright Vision Technologies; Oregon; Hybrid; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking a skilled MLOps Engineer to design, build, and operate high-performance inference platforms for large ML models in production. You will focus on systems engineering for AI deployment, including routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across diverse workloads.

The role requires strong distributed systems experience, proficiency in Python and a systems language (Go, Rust, or C++), and hands-on use of LLM inference

Qualifications

• Bachelor's or Master's degree in Computer Science or a related field.
• Six+ years of experience in distributed systems, infrastructure, or ML platform engineering.
• Strong proficiency in Python and a systems language such as Go, Rust, or C++.
• Deep experience operating high-throughput, low-latency services in production.
• Hands-on experience with LLM or large model inference frameworks such as vLLM or TensorRT-LLM.
• Strong understanding of GPU architecture, memory hierarchies, and accelerator utilization.
• Familiarity with Kubernetes, autoscaling, and modern cloud platforms.
• Experience with observability stacks including metrics, tracing, and structured logging.
• Solid grounding in performance engineering and capacity planning.
• Strong communication and incident response skills.

Responsibilities

• Design and operate model serving platforms for diverse workloads (LLMs, vision, recommendations).
• Optimize inference with batching, paged attention, speculative decoding, and multiplexing.
• Implement multi-tenant routing, rate limiting, and QoS across model endpoints.
• Build autoscaling and capacity management balancing latency, throughput and cost.
• Tune GPU utilization, memory, and KV cache for LLM workloads.
• Integrate serving with APIs, identity systems, and observability platforms.
• Implement caching, prompt deduplication, and response reuse strategies.
• Drive end-to-end observability including latency histograms and error tracking.
• Develop deployment workflows (canaries, shadow testing, rollbacks).
• Operate incident response for high-availability AI services.
• Collaborate with ML and product teams on model releases and rollouts.
• Implement security controls like request signing and content filtering.
• Document procedures and performance characteristics for internal teams.
• Stay current with AI serving research and translate to production.

Skills

Distributed systems

Python

Go

Rust

C++

LLM inference

TensorRT-LLM

GPU architecture

Kubernetes

Observability

Performance engineering

Incident response

Education

Bachelor's or Master’s in CS or related

Tools

Docker

Cloud platforms

Job description

Bright Vision Technologies is seeking a skilled MLOps Engineer to design, build, and operate high-performance inference platforms for large ML models in production. You will focus on systems engineering for AI deployment, including routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across diverse workloads.

The role requires strong distributed systems experience, proficiency in Python and a systems language (Go, Rust, or C++), and hands-on use of LLM inference