Senior ML Platform Engineer (Remote, Production Serving)

Bright Vision Technologies; Monroe Township (NJ); Remote; USD 100,000 - 160,000; Full time

Job summary

Bright Vision Technologies is seeking a ML Platform Engineer to design, build, and operate high-performance inference platforms for serving large machine learning models in production. The role focuses on the systems engineering side of AI deployment, including request routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across diverse model workloads.

The ideal candidate brings strong distributed systems and performance engineering expertise, has shipped

Qualifications

• 10+ years of experience in distributed systems, infrastructure, or ML platform engineering.
• Deep experience delivering high-throughput, low-latency services in production.
• Strong proficiency in Python and a systems language (Go, Rust, or C++).
• Experience with LLM/large-model inference frameworks and GPUs.

Responsibilities

• Design and operate model serving platforms for diverse workloads including LLMs and vision models.
• Optimize inference performance with batching, caching, and autoscaling.
• Implement multi-tenant routing, QoS, and capacity management.
• Tune GPU utilization and memory for efficient LLM serving workloads.
• Integrate with API gateways, observability stacks, and security controls.
• Develop deployment workflows with canary releases and automated rollback.

Skills

Distributed systems

Python

Go/C++

Kubernetes

Observability

LLM frameworks

GPU inference

Production experience

Strong communication

Education

Bachelor or Master in CS

Tools

vcLLM

TensorRT-LLM

Job description

Bright Vision Technologies is seeking a ML Platform Engineer to design, build, and operate high-performance inference platforms for serving large machine learning models in production. The role focuses on the systems engineering side of AI deployment, including request routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across diverse model workloads.

The ideal candidate brings strong distributed systems and performance engineering expertise, has shipped