Platform Engineer – AI Infra & Low-Latency APIs

Boson AI; Santa Clara (CA); On-site; USD 170,000 - 250,000; Full time

Job summary

Boson AI is seeking a backend infrastructure engineer to own and operate the core platform that powers our model APIs and agentic products. You will manage API serving, state management, data pipelines, and execution runtimes to ensure speed, reliability, and easy product integration.

Join a team of researchers and engineers delivering enterprise AI infrastructure with high performance, low latency, and scalable cloud deployments.

Qualifications

• 3+ years building and operating backend services in production.
• Strong knowledge of distributed systems, concurrency and fault tolerance.
• Experience with data pipelines and log processing using Kafka, Spark/Flink, or Airflow.
• Proficiency in at least one systems language (Go, Rust, Java, C++) or Python for performance-sensitive contexts.
• Experience with cloud infrastructure (AWS/GCP), Kubernetes, CI/CD, and on-call production.

Responsibilities

• Own and evolve core platform infrastructure including API serving and execution runtime.
• Design high-throughput, low-latency distributed services for model APIs.
• Build data pipelines for API logs, analytics, and billing.
• Develop internal SDKs and libraries with clear contracts for product teams.
• Architect memory and retrieval systems for conversational workloads.
• Instrument observability with SLIs/SLOs and structured logging.
• Collaborate with ML and product teams on model serving and tooling.

Skills

Distributed systems

API design

Cloud infrastructure

Tools

Kafka

Kinesis

Spark

Flink

Airflow

Go

Rust

Java

C++

Python

Kubernetes

AWS

GCP

Job description

Boson AI is seeking a backend infrastructure engineer to own and operate the core platform that powers our model APIs and agentic products. You will manage API serving, state management, data pipelines, and execution runtimes to ensure speed, reliability, and easy product integration.

Join a team of researchers and engineers delivering enterprise AI infrastructure with high performance, low latency, and scalable cloud deployments.