Real-Time ML Inference Lead - Low-Latency Architect

NTT DATA, Inc.; Charlotte (NC); On-site; USD 166,000 - 185,000; Full time

Job summary

NTT DATA, Inc. in Charlotte, NC seeks a Real-Time Inference Engineering Lead to design and industrialize low-latency, resilient model-serving services for predictive AI use cases.

You will define deployment patterns, capacity controls, monitoring, performance standards, and operational practices across cloud and on-premises environments. Ideal candidates have 8+ years in online inference, Kubernetes, and CI/CD, with experience in enterprise model-serving platforms and inference runtimes.

Qualifications

• 8+ years in online inference and real-time model-serving architecture.
• Experience with cloud and on-prem deployments.
• Degree in computer science or engineering or related field.

Responsibilities

• Architect low-latency online inference and real-time model-serving solutions.
• Develop scalable APIs, microservices, and deployment patterns for predictive models.
• Implement Kubernetes-based deployment, autoscaling, load balancing, and traffic-management strategies.
• Conduct benchmarking, performance tuning, capacity planning, and load testing.
• Define monitoring, alerting, SLOs, runbooks, and incident-response practices.
• Build CI/CD pipelines for repeatable model and service releases.
• Design resilience, failover, disaster recovery, and graceful-degradation patterns.
• Lead technical reviews and mentor inference and platform engineers.

Skills

Online inference

REST/gRPC APIs

Kubernetes

Autoscaling

Latency optimization

Monitoring & SLOs

CI/CD

Resilience & HA

Cloud & on-prem

Education

Degree in CS/Engineering

Tools

Model-serving platforms

Inference runtimes

Job description

NTT DATA, Inc. in Charlotte, NC seeks a Real-Time Inference Engineering Lead to design and industrialize low-latency, resilient model-serving services for predictive AI use cases.

You will define deployment patterns, capacity controls, monitoring, performance standards, and operational practices across cloud and on-premises environments. Ideal candidates have 8+ years in online inference, Kubernetes, and CI/CD, with experience in enterprise model-serving platforms and inference runtimes.