Senior Backend Engineer, Real-Time AI Infrastructure

Ellipsis Health; San Francisco (CA); Hybrid; USD 175,000 - 250,000; Full time

Job summary

Ellipsis Health is shaping healthcare through AI-driven voice technology in Silicon Valley. We are building production-grade back-ends for real-time voice pipelines, with streaming orchestration, low latency, and secure data handling.

The team focuses on extensible, observable software that scales with customer needs and regulatory requirements. Join a fast-growing company backed by top healthcare partners and perform in a highly collaborative environment across ML, product, and data engineering

Qualifications

• 4+ years building production back-ends in modern Python.
• Proven experience with real-time streaming systems—WebRTC, WebSockets, or gRPC streaming—and proficiency with asyncio, FastAPI, or similar async frameworks.
• Deep understanding of concurrency, buffering, audio codecs (Opus, PCM), and distributed tracing.
• Solid understanding of AWS/GCP/Azure, including container orchestration (Kubernetes/EKS/GKE), message queues (Kafka/SQS/Pub/Sub), and IaC (Terraform).
• Solid grasp of relational (PostgreSQL) and in-memory (Redis) data stores; able to model and persist conversational state.
• Excellent communication skills and a bias for measured, observable, and continuously deployable software.

Responsibilities

• Expand and maintain our real-time voice pipeline: design, implement, and maintain Python micro-services for conversational AI orchestration—audio capture, streaming transcription, prompt/LLM logic, synthesis, and playback.
• Integrate new providers & transports: add plug-ins for emerging ASR, TTS, LLM, and memory services; wire up WebRTC, SIP, or phone endpoints; build adapters that allow hot-swapping components without downtime. Build API endpoints.
• Deliver ultra-low latency (<500 ms round-trip): profile async pipelines (asyncio, FastAPI, gRPC), optimize buffering, concurrency, and back-pressure handling.
• Instrument & observe every hop: emit structured traces (OpenTelemetry), metrics, and logs for each pipeline stage; define SLOs for first-token latency, end-to-end latency, and streaming reliability.
• Harden for production: implement graceful retries, idempotent message passing, circuit breakers, and HIPAA-compliant security (encryption in transit, per-tenant isolation, secrets rotation).
• Collaborate cross-functionally with ML, product, data engineering, and client-SDK teams to deliver features such as voice cloning, multimodal hand-offs, and domain-specific memory retrieval.

Skills

Python

Asyncio

FastAPI

Real-time streaming

gRPC/WebRTC

Education

B.S./M.S. in CS, EE, or related fields

Tools

WebRTC

WebSockets

gRPC

Kubernetes

Terraform

Kafka

PostgreSQL

Redis

Job description

Ellipsis Health is shaping healthcare through AI-driven voice technology in Silicon Valley. We are building production-grade back-ends for real-time voice pipelines, with streaming orchestration, low latency, and secure data handling.

The team focuses on extensible, observable software that scales with customer needs and regulatory requirements. Join a fast-growing company backed by top healthcare partners and perform in a highly collaborative environment across ML, product, and data engineering