Staff Backend Engineer, AI Infra

Apollo; United States; Remote; USD 166,000 - 260,000; Full time

Job summary

A leading AI-native company in the United States is seeking an experienced platform engineer to design and implement scalable LLM infrastructure. The ideal candidate has over 8 years of experience in platform engineering and a strong background in AI infrastructure. This role offers competitive compensation with an annual pay range of $166,000 to $260,000 USD and extensive benefits including equity, flexible PTO, and health insurance.

Benefits

Equity

401(k) plan

Flex PTO

Employee assistance program

Qualifications

• 8+ years of experience in platform engineering, distributed systems, or AI infrastructure.
• Proven track record of designing large-scale production-grade systems.
• Deep understanding of software architecture principles.

Responsibilities

• Lead design and implementation of scalable LLM infrastructure.
• Architect and build frameworks for AI platform needs.
• Define and implement CI/CD best practices.

Skills

Platform engineering

AI infrastructure

Cloud platforms

Kubernetes

Python

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

FastAPI

Job description

Apollo.io is the leading go-to-market solution for revenue teams, trusted by over 500,000 companies and millions of users globally. Founded in 2015, Apollo.io provides sales and marketing teams with access to verified contact data for over 210 million B2B contacts and 35 million companies worldwide, along with tools to engage and convert these contacts in one unified platform. By helping revenue professionals find the most accurate contact information and automating the outreach process, Apollo.io turns prospects into customers. Apollo raised a series D in 2023 and is backed by top-tier investors, including Sequoia Capital, Bain Capital Ventures, and more, and counts the former President and COO of HubSpot, JD Sherman, among its board members.

Daily Adventures and Responsibilities

• Lead design and implementation of highly scalable, reliable LLM infrastructure that interfaces with multiple providers (Anthropic, OpenAI, GCP)
• Architect and build unified abstractions and frameworks for the organization's AI platform needs including:
  • Provider API abstractions, authentication systems, and sophisticated usage policies
  • Advanced provider routing with circuit breakers, retries, and graceful fallbacks
  • Implement guardrails, content filters, and standardized prompt templating systems
  • Design comprehensive observability for latency, cost, and quality metrics across AI systems
• Drive critical technical decisions across engineering organization for AI infrastructure adoption
• Lead development of core AI platform services from architecture to production, including developing robust async worker patterns and task orchestration systems
• Define and implement CI/CD best practices: establish pre-commit standards, reduce test flakiness, optimize test execution, harden GitHub Actions, and implement preview environments
• Architect cloud infrastructure and Kubernetes deployments with focus on security, scalability and cost efficiency
• Establish SLOs/SLIs and implement monitoring systems with Sentry, alert policies, metrics, logs, and dashboards
• Collaborate with cross-functional leadership to drive platform adoption and de-risk major launches
• Mentor engineers on platform best practices and guide architectural decisions across teams

Competencies

• Strategic thinking with ability to influence and align engineering decisions across the organization
• Exceptional communication skills for collaborating with engineering leadership, product, and business stakeholders
• Self-motivated with demonstrated ability to lead complex, multi-team initiatives
• Excellent analytical skills to evaluate and solve complex technical challenges
• Strong technical leadership with proven track record of mentoring engineers
• Demonstrated experience establishing engineering best practices and systems thinking
• Exceptional attention to detail and commitment to high-quality deliverables
• Ability to navigate ambiguity and drive technical clarity in complex problem spaces
• Proficiency in leveraging AI tools to enhance productivity and engineering workflows

Skills & Relevant Experience

Required:

• 8+ years of experience in platform engineering, distributed systems, or AI infrastructure
• Proven track record of designing and implementing large-scale, production-grade systems
• Experience leading technical initiatives that span multiple teams and organizations
• Deep understanding of software architecture principles and design patterns
• Strong knowledge of cloud platforms, container orchestration, and microservices
• Bachelor's degree in Computer Science, Engineering, or related technical field

Preferred:

• Experience building production AI/ML infrastructure and platforms
• Expertise with Python, FastAPI, and modern async programming patterns
• Strong background in Kubernetes, infrastructure as code, and cloud-native architectures
• Experience with observability systems, performance optimization, and reliability engineering
• Knowledge of security best practices for AI systems and data handling

The listed pay range reflects base salary range, except for sales roles. The range provided is the role’s On Target Earnings (OTE) range, meaning that the range includes both the sales commission/bonus targets and annual base salary for the role. This pay range may be inclusive of several career levels at Apollo and will be narrowed during the interview process based on factors including the candidate’s experience, qualifications, and location. Applicants outside the US may request the annual salary range for their location during the interview process.

Additional benefits for this role may include equity; company bonus or sales commissions/bonuses; 401(k) plan; at least 10 paid holidays per year, flex PTO, and parental leave; employee assistance program and wellbeing benefits; global travel coverage; life/AD&D/STD/LTD insurance; FSA/HSA and medical, dental, and vision benefits.

Annual Pay Range $166,000 — $260,000 USD. We are AI Native.

Apollo.io is an AI-native company built on a culture of continuous improvement. We’re on the front lines of driving productivity for our customers—and we expect the same mindset from our team. If you\'re energized by finding smarter, faster ways to get things done using AI and automation, you\'ll thrive here.

Why You’ll Love Working at Apollo

At Apollo, we’re driven by a shared mission: to help our customers unlock their full revenue potential. That’s why we take extreme ownership of our work, move with focus and urgency, and learn voraciously to stay ahead.

We invest deeply in your growth, ensuring you have the resources, support, and autonomy to own your role and make a real impact. Collaboration is at our core—we’re all for one, meaning you’ll have a team across departments ready to help you succeed. We encourage bold ideas and courageous action, giving you the freedom to experiment, take smart risks, and drive big wins.

If you’re looking for a place where your work matters, where you can push boundaries, and where your career can thrive—Apollo is the place for you.

Learn more here!