Real-Time Inference Platform Lead

NTT DATA, Inc.; Charlotte (NC); Hybrid; USD 84,000 - 125,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

401k with company match

Paid time off

Employee assistance program

Flexible spending or health savings

Job summary

NTT DATA Services in Charlotte, NC seeks a Real-Time Inference Engineering Lead (FTE / Hybrid) to design, build, and industrialize low-latency, resilient model-serving services for real-time predictive AI use cases. The role combines hands-on engineering with technical leadership across online inference architecture, deployment patterns, APIs, and operations.

The successful candidate will establish reusable patterns enabling multiple teams to deploy and operate predictive models safely and at

Qualifications

• 8+ years of software engineering, platform engineering, or related infra experience.
• 4+ years of building or operating production APIs, distributed systems, or real-time ML workloads.
• Experience delivering highly available, performance-sensitive services.

Responsibilities

• Define target architecture and standards for real-time model-serving across cloud and on-premises.
• Design, build, test, deploy, and operate scalable online inference services with low latency.
• Establish reusable model-serving patterns for synchronous and asynchronous inference.
• Develop deployment approaches for model packaging, versioning, and canary deployments.
• Design secure API patterns for inference services, including auth, authz, and rate limiting.
• Lead Kubernetes-based serving platforms using GKE and OpenShift.
• Implement autoscaling, capacity planning, and workload isolation for variable demand.
• Establish monitoring, dashboards, SLIs/SLOs, and incident-management practices.

Skills

Online inference architecture

Model-serving patterns

REST APIs

Kubernetes

CI/CD

Observability

Security patterns

Performance optimization

Distributed systems

Tools

GKE

OpenShift

Job description

NTT DATA Services in Charlotte, NC seeks a Real-Time Inference Engineering Lead (FTE / Hybrid) to design, build, and industrialize low-latency, resilient model-serving services for real-time predictive AI use cases. The role combines hands-on engineering with technical leadership across online inference architecture, deployment patterns, APIs, and operations.

The successful candidate will establish reusable patterns enabling multiple teams to deploy and operate predictive models safely and at