Real-Time Inference Platform Lead

Creative Solutions Services, LLC; Charlotte (NC); On-site; USD 80,000 - 89,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

Health Savings Account (HSA)

AD&D insurance

401(k) program

Job summary

NTT DATA seeks an experienced Real-Time Inference Engineering Lead in Charlotte, NC to design and industrialize low-latency model-serving services for predictive AI use cases. You will define deployment patterns, capacity controls, monitoring, and operational practices across cloud and on-prem environments.

The role requires leading the architectural direction, building scalable APIs, and ensuring resilient, scalable real-time inference platforms while mentoring a team of inference and platform

Qualifications

• 8+ years of experience in online inference and real-time model-serving architecture.
• REST/gRPC APIs and distributed microservices.
• Kubernetes, containers, autoscaling, and traffic management.
• Performance engineering, latency optimization, and load testing.
• Monitoring, SLOs, capacity planning, and production operations.
• CI/CD and progressive-deployment approaches.
• Resilience and high-availability engineering.
• Cloud and on-premises deployment experience.
• Degree in computer science, engineering, or a related discipline.
• Experience with enterprise model-serving platforms and inference runtimes.
• Cloud, Kubernetes, SRE, or ML engineering certification.

Responsibilities

• Architect low-latency online inference and real-time model-serving solutions.
• Develop scalable APIs, microservices, and deployment patterns for predictive models.
• Implement Kubernetes-based deployment, autoscaling, load balancing, and traffic-management strategies.
• Conduct benchmarking, performance tuning, capacity planning, and load testing.
• Define monitoring, alerting, SLOs, runbooks, and incident-response practices.
• Build CI/CD pipelines for repeatable model and service releases.
• Design resilience, failover, disaster recovery, and graceful-degradation patterns.
• Lead technical reviews and mentor inference and platform engineers.

Skills

Real-Time Inference

REST/gRPC APIs

Kubernetes

Performance engineering

Monitoring/SLOs

CI/CD

Resilience & HA

Cloud & on-premises

Education

Degree in CS/Engineering

Tools

Enterprise model-serving platforms

Job description

NTT DATA seeks an experienced Real-Time Inference Engineering Lead in Charlotte, NC to design and industrialize low-latency model-serving services for predictive AI use cases. You will define deployment patterns, capacity controls, monitoring, and operational practices across cloud and on-prem environments.

The role requires leading the architectural direction, building scalable APIs, and ensuring resilient, scalable real-time inference platforms while mentoring a team of inference and platform