Senior ML Model Serving Engineer (Remote)

NEPSE Trading; United States; Hybrid; USD 74,000 - 98,000; Full time

Job summary

Bright Vision Technologies is seeking a Model Serving Engineer to design and operate high-performance inference platforms for production ML models. This remote role focuses on scalable serving systems and end-to-end observability.

You will work on routing, batching, caching, autoscaling, and GPU utilization, with emphasis on latency, throughput, and cost efficiency. 7+ years of distributed systems experience required.

Qualifications

• Bachelor’s or Master’s degree in Computer Science or a related field.
• 7+ years of experience in distributed systems, infrastructure, or ML platform engineering.
• Strong proficiency in Python and at least one systems language (Go, Rust, or C++).
• Deep experience operating high-throughput, low-latency services in production.
• Hands-on experience with LLM or large model inference frameworks such as vLLM or TensorRT-LLM.
• Strong understanding of GPU architecture and accelerator utilization.
• Familiarity with Kubernetes, autoscaling, and cloud platforms.
• Experience with observability stacks (metrics, tracing, logging).
• Solid grounding in performance engineering and capacity planning.

Responsibilities

• Design, build, and operate high-performance inference platforms for serving ML models.
• Focus on systems engineering aspects of AI deployment including routing, batching, and caching.
• Implement autoscaling, GPU utilization optimization, and end-to-end observability.
• Collaborate across teams to balance latency, throughput, and cost.

Skills

Python

Go

Rust

C++

Distributed systems

LLM inference frameworks

Kubernetes

Cloud platforms

Observability

Performance engineering

GPU utilization

Auto-scaling

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

vLLM

TensorRT-LLM

Prometheus

Jaeger

Job description

Bright Vision Technologies is seeking a Model Serving Engineer to design and operate high-performance inference platforms for production ML models. This remote role focuses on scalable serving systems and end-to-end observability.

You will work on routing, batching, caching, autoscaling, and GPU utilization, with emphasis on latency, throughput, and cost efficiency. 7+ years of distributed systems experience required.