Senior AI Platform Engineer: LLM & Agent Infra

Coinbase; Charlotte (NC); Hybrid; USD 186,000 - 219,000; Full time

Job summary

Coinbase is seeking a Senior Software Engineer for the AI Platform to build and operate the LLM and agent infrastructure used across the company. You will own core platform systems including the LLM gateway, AI hub, and agent runtime, ensuring scalability, governance, and observable performance.

You will lead multi-quarter initiatives, collaborate with security, legal, product, and cloud partners, and implement FinOps capabilities to optimize costs.

Qualifications

• 5+ years of software engineering experience building ML/AI infrastructure, LLM systems, or distributed platform services at scale.
• Demonstrated ability to lead multi-quarter initiatives end-to-end, with measurable platform-wide impact.
• Deep expertise in LLM orchestration patterns, agent frameworks, vector databases, and model serving infrastructure across multiple providers.
• Proven track record of cross-functional partnership with security, legal, and product teams to ship governed, production-grade AI systems.
• Fluency in Python and at least one systems language (Go, Rust, or C), with production experience building APIs, microservices, and runtime environments.
• Utilizes generative AI responsibly, maintaining human oversight to deliver business-ready outputs and drive measurable improvements in workflow efficiency, cost, and quality.

Responsibilities

• Own the architecture and delivery of core platform systems including the LLM Gateway, AI Hub, and agent runtime with microVM sandboxes and governed MCP gateway.
• Drive the design and implementation of Knowledge Base infrastructure, connecting data sources to auto-provisioned vector and markdown stores queryable by any agent.
• Lead AI FinOps capabilities including spend attribution, governance, and cost optimization across all AI workloads company-wide.
• Partner across engineering, security, legal, finance, product, and external partners to ship high-impact platform capabilities.
• Build evaluation and observability tooling including LLM-as-judge harnesses, full tracing, and feedback loops for production agents.
• Shape applied AI strategy by embedding with teams to build production agents, applying fine-tuning and traditional ML where it outperforms prompting.

Skills

ML/AI infra

Python

Go

Rust

C

LLM systems

APIs & microservices

Tools

LLM Gateway

Vector databases

MicroVM sandboxes

Job description

Coinbase is seeking a Senior Software Engineer for the AI Platform to build and operate the LLM and agent infrastructure used across the company. You will own core platform systems including the LLM gateway, AI hub, and agent runtime, ensuring scalability, governance, and observable performance.

You will lead multi-quarter initiatives, collaborate with security, legal, product, and cloud partners, and implement FinOps capabilities to optimize costs.