Senior ML Infra Engineer — Remote AI Serving

United States Digital Space LLC; United States; Remote; USD 90,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking a Machine Learning Infrastructure Engineer to design, build, and operate high-performance inference platforms for serving large ML models in production. This remote role focuses on systems engineering for AI deployment—routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across varied model workloads.

Candidates should have distributed systems expertise and a track record shipping scalable serving systems.

Qualifications

• Six or more years of experience in distributed systems, infrastructure, or ML platform engineering.
• Strong proficiency in Python and a systems language (Go, Rust, or C++).
• Deep experience operating high-throughput, low-latency services in production.
• Hands-on experience with LLM inference frameworks such as vLLM or TensorRT-LLM.
• Strong understanding of GPU architecture, memory hierarchies, and accelerator utilization.
• Familiarity with Kubernetes, autoscaling, and modern cloud platforms.
• Experience with observability stacks including metrics, tracing, and structured logging.
• Solid grounding in performance engineering and capacity planning.
• Strong communication and incident response skills.

Responsibilities

• Design and operate model serving platforms for diverse workloads including LLMs and vision models.
• Optimize inference performance using batching, attention mechanisms, and request multiplexing.
• Implement multi-tenant routing, rate limiting, and QoS across endpoints.
• Build autoscaling and capacity management balancing latency, throughput, and cost.
• Tune GPU utilization and memory management for LLM workloads.
• Integrate serving with API gateways, identity, and observability stacks.
• Develop deployment workflows with canaries, shadow testing, and rollbacks.
• Handle incident response and drive reliability improvements.

Skills

Distributed systems

Python

Go

Rust

C++

LLM inference

GPU knowledge

Kubernetes

Observability

Performance engineering

Tools

vLLM

TensorRT-LLM

Job description

Bright Vision Technologies is seeking a Machine Learning Infrastructure Engineer to design, build, and operate high-performance inference platforms for serving large ML models in production. This remote role focuses on systems engineering for AI deployment—routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across varied model workloads.

Candidates should have distributed systems expertise and a track record shipping scalable serving systems.