Senior Backend Platform Engineer — Enterprise AI Systems

Boson AI; Palo Alto (CA); On-site; USD 150,000 - 400,000; Full time

Job summary

Boson AI in Palo Alto, CA seeks a backend platform engineer to own the infrastructure behind model APIs and agentic products. Responsibilities include API serving, state management, data pipelines, and memory systems for low-latency workloads.

You will collaborate with ML and product teams to deliver scalable, reliable platform services in a cloud/K8s environment, focusing on performance, correctness, and developer experience.

Qualifications

• 3+ years building and operating backend systems at scale.
• Strong distributed systems fundamentals: concurrency, fault tolerance, consistency tradeoffs.
• Hands-on experience with data pipeline infrastructure (Kafka/Kinesis, Spark/Flink, Airflow).
• Track record of designing APIs and frameworks used by multiple teams.

Responsibilities

• Own and evolve the core platform infrastructure: API serving layer, state management, policy enforcement engine, and execution runtime for agentic workflows.
• Design and operate high-throughput, low-latency distributed services backing model APIs, including routing, rate limiting, and multi-tenant isolation.
• Build and maintain downstream data pipelines (ETL/ELT) for API logs, usage analytics, and billing.
• Develop production-grade internal SDKs and libraries with clean APIs and strong type safety.
• Architect context and memory systems for conversational workloads with vector stores and retrieval pipelines.
• Instrument end-to-end observability: SLIs/SLOs, structured logging, tracing, reliability improvements.
• Collaborate with ML and product teams to integrate model serving and tooling infrastructure.

Skills

Backend systems

Distributed systems

Data pipelines

APIs design

Go

Kubernetes

Tools

Kafka

Airflow

Kubernetes

Spark

Job description

Boson AI in Palo Alto, CA seeks a backend platform engineer to own the infrastructure behind model APIs and agentic products. Responsibilities include API serving, state management, data pipelines, and memory systems for low-latency workloads.

You will collaborate with ML and product teams to deliver scalable, reliable platform services in a cloud/K8s environment, focusing on performance, correctness, and developer experience.