Real-Time ML Inference Lead — Latency & Platform Architect

Stellent IT LLC; Charlotte (NC); Hybrid; USD 96,000 - 138,000; Full time

Job summary

Stellent IT LLC in Charlotte, North Carolina is seeking an Inference Lead (Machine Learning Platform Engineer Lead Real-Time Inference) for a six-month contract. The role focuses on designing low-latency, resilient model-serving services for predictive AI use cases, with cloud and on-prem deployment.

You will define deployment patterns, implement Kubernetes-based operations, develop scalable APIs, and lead a team of inference and platform engineers while ensuring performance, reliability, and

Qualifications

• Design and implement low-latency online inference and real-time model-serving systems.
• Experience building scalable APIs and microservices for predictive models.
• Proficiency with Kubernetes, autoscaling, and traffic management.
• Ability to perform benchmarking, latency optimization, and load testing.
• Strong focus on monitoring, SLOs, capacity planning, and production operations.
• Experience with CI/CD and progressive deployment approaches.
• Design for resilience, failover, and disaster recovery.

Responsibilities

• Architect low-latency online inference and real-time model-serving solutions.
• Develop scalable APIs, microservices, and deployment patterns for predictive models.
• Implement Kubernetes-based deployment, autoscaling, load balancing, and traffic-management strategies.
• Conduct benchmarking, performance tuning, capacity planning, and load testing.
• Optimize latency, throughput, resource consumption, availability, and cost.
• Define monitoring, alerting, SLOs, runbooks, and incident-response practices.
• Build CI/CD pipelines for repeatable model and service releases.
• Design resilience, failover, disaster recovery, and graceful-degradation patterns.
• Lead technical reviews and mentor inference and platform engineers.

Skills

Low-latency inference

Kubernetes

REST/gRPC APIs

CI/CD pipelines

Performance tuning

Monitoring & SLOs

Cloud and on-prem deployments

Resilience & high-availability

Distributed microservices

Education

Degree in CS/Engineering or related field

Tools

Kubernetes

Containers

Job description

Stellent IT LLC in Charlotte, North Carolina is seeking an Inference Lead (Machine Learning Platform Engineer Lead Real-Time Inference) for a six-month contract. The role focuses on designing low-latency, resilient model-serving services for predictive AI use cases, with cloud and on-prem deployment.

You will define deployment patterns, implement Kubernetes-based operations, develop scalable APIs, and lead a team of inference and platform engineers while ensuring performance, reliability, and