Senior AI Inference Platform Lead

Akamai Technologies, Inc.; City of Albany (NY); Hybrid; USD 146,000 - 264,000; Full time

Benefits offered by this job

Healthcare

401K

Paid vacation and holidays

Parental leave

Employee assistance program

Job summary

Akamai Technologies, Inc. is seeking a Senior II Software Engineer Lead to drive the design and implementation of the Akamai Inference Cloud.

You will lead critical components, ensure performance and scalability, and guide engineers across regions to deliver OpenAI-compatible endpoints for AI workloads. You will mentor teams, decide on tooling, optimize containerized AI workloads, and collaborate with cross-functional partners to define standards and operational excellence.

Qualifications

• Bachelor’s degree or equivalent required.
• Experience with distributed systems, cloud services, and platform engineering.
• Hands-on AI inference, model serving, or LLM deployment.
• Working knowledge of inference frameworks (TensorRT, vLLM, TorchServe, Triton).
• Experience with containerization/orchestration (Kubernetes, Docker) and cloud-native architectures.
• Strong DevOps practices, CI/CD pipelines, and infrastructure-as-code.
• Proven technical leadership through mentoring and code reviews.
• Familiarity with API design (REST/gRPC) and GPU hardware acceleration.
• Skills in observability, monitoring, and debugging of distributed systems.

Responsibilities

• Lead design and implementation of core platform components for Akamai Inference Cloud.
• Drive technical decisions for AI inference workloads and choose appropriate tools/frameworks.
• Mentor engineers through code reviews and design discussions.
• Implement and optimize containerized AI workloads with hardware-specific optimizations.
• Collaborate across teams to define requirements and uphold operational excellence.
• Ensure performance, scalability, and reliability of the inference platform.

Skills

Distributed systems

Cloud services

Platform engineering

AI inference

Model serving

LLM deployment

CI/CD pipelines

API design

Observability

Technical leadership

Education

Bachelor’s degree or equivalent

Tools

Kubernetes

Docker

Job description

Akamai Technologies, Inc. is seeking a Senior II Software Engineer Lead to drive the design and implementation of the Akamai Inference Cloud.

You will lead critical components, ensure performance and scalability, and guide engineers across regions to deliver OpenAI-compatible endpoints for AI workloads. You will mentor teams, decide on tooling, optimize containerized AI workloads, and collaborate with cross-functional partners to define standards and operational excellence.