Senior Software Engineer - Systems

Boson AI; Santa Clara (CA); On-site; USD 170,000 - 250,000; Full time

Job summary

Boson AI is seeking a backend infrastructure engineer to own and operate the core platform that powers our model APIs and agentic products. You will manage API serving, state management, data pipelines, and execution runtimes to ensure speed, reliability, and easy product integration.

Join a team of researchers and engineers delivering enterprise AI infrastructure with high performance, low latency, and scalable cloud deployments.

Qualifications

• 3+ years building and operating backend services in production.
• Strong knowledge of distributed systems, concurrency and fault tolerance.
• Experience with data pipelines and log processing using Kafka, Spark/Flink, or Airflow.
• Proficiency in at least one systems language (Go, Rust, Java, C++) or Python for performance-sensitive contexts.
• Experience with cloud infrastructure (AWS/GCP), Kubernetes, CI/CD, and on-call production.

Responsibilities

• Own and evolve core platform infrastructure including API serving and execution runtime.
• Design high-throughput, low-latency distributed services for model APIs.
• Build data pipelines for API logs, analytics, and billing.
• Develop internal SDKs and libraries with clear contracts for product teams.
• Architect memory and retrieval systems for conversational workloads.
• Instrument observability with SLIs/SLOs and structured logging.
• Collaborate with ML and product teams on model serving and tooling.

Skills

Distributed systems

API design

Cloud infrastructure

Tools

Kafka

Kinesis

Spark

Flink

Airflow

Go

Rust

Java

C++

Python

Kubernetes

AWS

GCP

Job description

About Boson AI

At Boson AI, we are not just building AI solutions; we are pioneering the future of enterprise AI. Driven by a passion for cutting‑edge AI research, particularly in the transformative areas of large language models and agentic systems, our mission is to tackle the most complex real‑world problems for businesses and unlock significant value. We are a dynamic and collaborative team of researchers and engineers who thrive on pushing the boundaries of what’s possible, dedicated to delivering high‑quality, reliable products that seamlessly integrate into the fabric of enterprise workflows and set new industry standards.

About the Role

Build and operate the core platform behind Boson's model APIs and agentic products. You’ll own the infrastructure that every Boson agent runs on — API serving, state management, data pipelines, context retrieval, and execution runtime — and make it fast, reliable, and easy for product teams to build on.

Responsibilities

• Own and evolve the core platform infrastructure: API serving layer, state management, policy enforcement engine, and execution runtime for agentic workflows
• Design and operate high‑throughput, low‑latency distributed services that back our model API products — including request routing, load management, rate limiting, and multi‑tenant isolation
• Build and maintain downstream data pipelines (ETL/ELT) for API logs, usage analytics, and billing — ensuring data correctness, freshness, and queryability at scale
• Develop production‑grade internal SDKs and libraries with clean APIs, strong type safety, and clear contracts that product teams can build on confidently
• Architect context and memory systems for conversational workloads — low‑latency retrieval, caching, and integration with vector stores and retrieval pipelines
• Instrument end‑to‑end observability: define SLIs/SLOs, build structured logging and tracing, and drive reliability improvements across the platform
• Collaborate closely with ML and product teams to integrate model serving, voice runtime, and tooling infrastructure under tight latency and quality constraints

Qualifications

• 3+ years building and operating backend systems at scale — you've owned services that other teams depend on in production
• Strong distributed systems fundamentals: concurrency, fault tolerance, consistency tradeoffs, capacity planning
• Hands‑on experience with data pipeline infrastructure (Kafka/Kinesis, Spark/Flink, Airflow, or similar) for log processing, analytics, or ETL workloads
• Track record of designing APIs and frameworks adopted by other engineering teams — you care about developer experience and long‑term maintainability
• Proficiency in at least one systems language (Go, Rust, Java, C++) or Python in a performance‑sensitive context
• Comfortable working across the stack: cloud infrastructure (AWS/GCP), containerized deployments (K8s), CI/CD, and production oncall

Bonus points

• Experience with LLM serving, agentic orchestration patterns (ReAct, planner‑executor), or RAG pipelines
• Familiarity with emerging agent integration protocols (MCP, A2A) or orchestration frameworks (LangChain, LlamaIndex)
• Background in real‑time media systems (audio/video streaming, low‑latency signaling)
• Experience building high‑stakes platform services (payments, identity, core data) where correctness and auditability are non‑negotiable

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.