Remote ML Platform Engineer — High-Performance Inference

Bright Vision Technologies; Monroe (NJ); Remote; USD 100,000 - 160,000; Full time

Job summary

Bright Vision Technologies in the United States is seeking an ML Platform Engineer to design, build, and operate high-performance inference platforms for serving large ML models in production. This role emphasizes systems engineering of AI deployment—request routing, batching, autoscaling, GPU utilization, and end-to-end observability across diverse model workloads.

Candidates should have strong distributed systems experience and a track record shipping serving systems at scale, balancing

Qualifications

• Bachelor's or Master's degree in Computer Science or related field.
• 10+ years of experience in distributed systems, infrastructure, or ML platform engineering.
• Strong proficiency in Python and systems languages (Go, Rust, or C++).
• Experience with high-throughput, low-latency production services.
• Hands-on experience with LLM inference frameworks (vcLLM, TensorRT-LLM).
• Familiarity with Kubernetes and modern cloud platforms.
• Observability with metrics, tracing, and structured logging.
• Strong communication and incident response skills.

Responsibilities

• Design and operate model serving platforms supporting diverse workloads including LLMs, vision models, and recommendations.
• Optimize inference performance using batching, paged attention, and queueing strategies.
• Implement multi-tenant routing, rate limiting, and QoS across endpoints.
• Build autoscaling and capacity management to balance latency, throughput and cost.
• Tune GPU utilization, memory management, and KV cache strategies for LLMs.
• Integrate serving with API gateways, identity systems, and observability tools.
• Implement caching and response reuse strategies where appropriate.
• Drive end-to-end observability including latency histograms and GPU utilization.
• Develop deployment workflows with canary releases and rollbacks.
• Operate incident response for high-availability AI services and reliability improvements.
• Collaborate with ML and product teams on new model releases.
• Implement security controls and documentation for internal teams.
• Stay current with AI serving research and translate advances into production.

Skills

Python

Go

Rust

C++

Distributed systems

Education

Bachelor's or Master's degree in Computer Science or related field

Tools

Kubernetes

TensorRT-LLM

vcLLM

Cloud platforms

Job description

Bright Vision Technologies in the United States is seeking an ML Platform Engineer to design, build, and operate high-performance inference platforms for serving large ML models in production. This role emphasizes systems engineering of AI deployment—request routing, batching, autoscaling, GPU utilization, and end-to-end observability across diverse model workloads.

Candidates should have strong distributed systems experience and a track record shipping serving systems at scale, balancing