Remote ML Platform Engineer: High-Performance Inference

Bright Vision Technologies; SOUTH PLAINFIELD (NJ); Remote; USD 100,000 - 160,000; Full time

Job summary

Bright Vision Technologies is seeking an experienced ML Platform Engineer to design, build, and operate high-performance inference platforms for production ML models.

This is a 100% remote, full-time role focused on distributed systems, performance engineering, autoscaling, GPU utilization, and end-to-end observability across diverse workloads.

You will collaborate with ML and product teams to deploy new model releases, optimize cost and latency, and drive reliability at scale.

Qualifications

• 10+ years of experience in distributed systems or ML platform engineering.
• Strong Python and systems programming skills.
• Hands-on with model serving frameworks and GPU utilization.
• Experience with Kubernetes and cloud platforms.
• Observability, metrics, tracing, and logging expertise.

Responsibilities

• Design and operate ML model serving platforms for diverse workloads.
• Optimize inference performance via batching, caching, and routing.
• Implement multi-tenant routing, rate limiting, and QoS policies.
• Build autoscaling and capacity management to balance latency and cost.
• Tune GPU utilization and memory for serving workloads.
• Integrate serving with API gateways and observability platforms.
• Develop deployment workflows including canary releases and rollbacks.
• Maintain high-availability and incident response for AI services.
• Collaborate with ML and product teams on model releases.

Skills

Python

Go

Rust

C++

Distributed systems

Observability

Cloud platforms

Performance tuning

Incident response

Communication

Education

Bachelor's or Master's degree in Computer Science

Tools

Kubernetes

TensorRT-LLM

vcLLM

Job description

Bright Vision Technologies is seeking an experienced ML Platform Engineer to design, build, and operate high-performance inference platforms for production ML models.

This is a 100% remote, full-time role focused on distributed systems, performance engineering, autoscaling, GPU utilization, and end-to-end observability across diverse workloads.

You will collaborate with ML and product teams to deploy new model releases, optimize cost and latency, and drive reliability at scale.