Staff Infra Engineer: AI-Scale Cloud & Kubernetes

LlamaIndex, Inc.; San Francisco (CA); Hybrid; USD 200,000 - 275,000; Full time

Benefits offered by this job

Competitive base salary and equity

Comprehensive medical/dental/vision

Unlimited paid time off

Daily catered lunch and snacks

Job summary

LlamaIndex, Inc. in San Francisco is seeking an experienced Infra engineer to design, build, and scale core infrastructure powering a high-volume data platform for AI applications.

You will manage cloud resources, Kubernetes clusters, and deployment tooling, collaborating with multiple engineering teams to support rapid growth and diverse deployment models. This role emphasizes cost efficiency, security, and observability.

Qualifications

• 8+ years of engineering experience.
• Experience on Platform or Infrastructure teams with major projects involving infrastructure components (Terraform/CDKTF, Kubernetes, Helm, test infrastructure, release management, observability).
• Experience in optimizing cloud resource utilization.
• Proficient in tuning Kubernetes clusters and cloud resources for cost and performance efficiency.
• Willing to build LlamaIndex’s engineering culture as we grow.
• You can balance speed and pragmatism and build the appropriate solutions for each stage of the company’s growth.
• Hands-on proficiency with modern LLM tooling and production AI systems, including model APIs, agent or RAG frameworks, evaluation and tracing tools, and the operational characteristics of LLM workloads.

Responsibilities

• Collaborate with other engineering teams to build and maintain foundational systems that empower developers and support rapid growth.
• Design and implement scalable infrastructure solutions for deployment models including SaaS, single-tenant, and private deployments.
• Manage and optimize cloud resources and Kubernetes clusters for cost-effectiveness and performance.
• Enable external customer deployment success through maintaining clear infrastructure boundaries and principles.
• Optimize and improve the release and deployment processes to enhance efficiency and reliability.
• Ensure compliance with relevant regulations and implement robust security measures across environments.
• Build and operate infrastructure for production LLM applications, including model integrations, inference workloads, evaluation pipelines, observability, and agentic workflows.

Skills

Platform/infrastructure experience

Solving complex systems

Balancing speed and pragmatism

LLM tooling familiarity

Tools

Terraform/CDKTF

Kubernetes

Helm

Observability

Release management

Job description

LlamaIndex, Inc. in San Francisco is seeking an experienced Infra engineer to design, build, and scale core infrastructure powering a high-volume data platform for AI applications.

You will manage cloud resources, Kubernetes clusters, and deployment tooling, collaborating with multiple engineering teams to support rapid growth and diverse deployment models. This role emphasizes cost efficiency, security, and observability.