Founding AI Voice Systems Engineer (Real-Time Eval)

Roark; United States; Remote; USD 150,000 - 190,000; Full time

Job summary

Roark is hiring a founding-level engineer to build evaluation and measurement infrastructure for an AI voice calling platform. You’ll score production calls in near real time, layer audio-native signals on top of LLM evaluators, and power a realistic call simulation engine.

You’ll own large-scale analytics, distributed workflows, and the dashboards that monitor production metrics, with emphasis on correctness, replay safety, and ground-truth validation.

Qualifications

• Four or more years of production experience in backend, ML, or audio systems.
• Hands-on depth in at least one: speech/audio processing, LLM evaluation pipelines, or high-volume data infra.
• Strong drive for correctness with idempotency and ground-truth labeling.
• Willing to own operations like dashboards and alerts, and communicate clearly.

Responsibilities

• Design and operate evaluation pipelines scoring large volumes of production calls with low latency.
• Build audio-native metrics validated against human ground truth and LLm evaluators.
• Extend a simulation engine dialing real agents across phone/WebRTC with varied personas and noise.
• Own analytics infra: columnar stores, key-value streams, dashboards, and queries.
• Orchestrate long-running workflows (replays, backfills, load tests) with durability and correctness.
• Operate dashboards, alerts, and root-cause analysis when metrics drift.

Skills

Backend development

ML systems

Audio processing

TypeScript / Python

Tools

Temporal

PostgreSQL

DynamoDB

ClickHouse

Job description

Roark is hiring a founding-level engineer to build evaluation and measurement infrastructure for an AI voice calling platform. You’ll score production calls in near real time, layer audio-native signals on top of LLM evaluators, and power a realistic call simulation engine.

You’ll own large-scale analytics, distributed workflows, and the dashboards that monitor production metrics, with emphasis on correctness, replay safety, and ground-truth validation.