Founding AI Voice Systems Engineer: Real-Time Evaluation

Roark; United States; Remote; USD 150,000 - 230,000; Full time

Job summary

Roark is seeking a founding-level engineer to build the evaluation and measurement infrastructure for an AI voice calling platform. You will score production calls in near real time, layer audio-native signals on top of LLM evaluators, and power a simulation engine that places realistic automated calls.

You will own large-scale analytics and distributed workflows, with dashboards, alerts, and post-mortems to ensure reliability and correctness across the system.

Qualifications

• Four or more years of production experience in backend, ML, or audio systems.
• Depth in at least one: speech/audio processing, LLM evaluation pipelines, or high-volume data infrastructure.
• Strong commitment to correctness, idempotency, and ground-truth labels.
• Willingness to own operations including dashboards and alerts.

Responsibilities

• Design and operate evaluation pipelines that score large volumes of production calls with low latency.
• Build audio-native metrics and validate them against ground truth alongside LLM evaluators.
• Extend a simulation engine to dial real agents across phone and WebRTC with varied personas.
• Own analytics infrastructure including columnar stores, streams, and dashboards.
• Orchestrate long-running workflows with a durable engine and ensure idempotent behavior.
• Operate dashboards, alerts, and perform root-cause analysis when metrics drift.

Skills

Backend development

ML systems

Audio processing

Python/TypeScript

Data infrastructure

Operational ownership

Technical writing

Tools

TypeScript

Bun

AWS (SST)

ClickHouse

DynamoDB

Temporal

Postgres

WebRTC

Job description

Roark is seeking a founding-level engineer to build the evaluation and measurement infrastructure for an AI voice calling platform. You will score production calls in near real time, layer audio-native signals on top of LLM evaluators, and power a simulation engine that places realistic automated calls.

You will own large-scale analytics and distributed workflows, with dashboards, alerts, and post-mortems to ensure reliability and correctness across the system.