Senior ML Platform Engineer - Remote, Scalable Inference

Bright Vision Technologies; North Brunswick Township (NJ); À distance; USD 100 000 - 160 000; Plein temps

Résumé du poste

Bright Vision Technologies is seeking a senior ML Platform Engineer to design, build, and operate high-performance inference platforms for serving large machine learning models in production. The role emphasizes systems engineering for AI deployment, including routing, batching, caching, autoscaling, GPU utilization, and observability across diverse workloads.

The ideal candidate has deep expertise in distributed systems, has shipped serving systems at scale, and can balance latency, throughput,

Qualifications

• Bachelor’s or Master’s degree in Computer Science or a related field.
• 10+ years of experience in distributed systems, infrastructure, or ML platform engineering.
• Strong proficiency in Python and a systems language (Go, Rust, or C++).
• Extensive production experience with high-throughput, low-latency services.
• Hands-on experience with LLM inference frameworks such as vcLLM or TensorRT-LLM.
• Knowledge of GPU architecture, memory hierarchies, and accelerator utilization.
• Experience with Kubernetes, autoscaling, and modern cloud platforms.
• Familiarity with observability stacks (metrics, tracing, logging).
• Excellent communication and incident response skills.

Responsabilités

• Design and operate model serving platforms supporting diverse workloads including LLMs, vision models, and recommendation systems.
• Optimize inference performance using batching, paging, and speculative decoding.
• Implement multi-tenant routing, rate limiting, and quality-of-service policies across endpoints.
• Build autoscaling and capacity management systems balancing latency, throughput, and cost.
• Tune GPU utilization, memory management, and KV cache strategies for LLM serving workloads.
• Integrate model serving with API gateways, identity systems, and observability platforms.
• Develop deployment workflows with canary releases and automated rollback.
• Operate incident response for high-availability AI services and drive reliability improvements.
• Collaborate with ML and product teams on model releases and capability rollouts.
• Document procedures and performance characteristics for internal teams.

Connaissances

Python

Go

Rust

C++

Kubernetes

Observability

Performance engineering

Incident response

Distributed systems

Formation

Bachelor’s or Master’s in CS

Outils

vcLLM

TensorRT-LLM

Description du poste

Bright Vision Technologies is seeking a senior ML Platform Engineer to design, build, and operate high-performance inference platforms for serving large machine learning models in production. The role emphasizes systems engineering for AI deployment, including routing, batching, caching, autoscaling, GPU utilization, and observability across diverse workloads.

The ideal candidate has deep expertise in distributed systems, has shipped serving systems at scale, and can balance latency, throughput,