Founding Software Engineer — AI & Voice Systems

Roark; USA; Remote; USD 150.000 - 190.000; Vollzeit

Zusammenfassung

Roark is hiring a founding-level engineer to build evaluation and measurement infrastructure for an AI voice calling platform. You’ll score production calls in near real time, layer audio-native signals on top of LLM evaluators, and power a realistic call simulation engine.

You’ll own large-scale analytics, distributed workflows, and the dashboards that monitor production metrics, with emphasis on correctness, replay safety, and ground-truth validation.

Qualifikationen

• Four or more years of production experience in backend, ML, or audio systems.
• Hands-on depth in at least one: speech/audio processing, LLM evaluation pipelines, or high-volume data infra.
• Strong drive for correctness with idempotency and ground-truth labeling.
• Willing to own operations like dashboards and alerts, and communicate clearly.

Aufgaben

• Design and operate evaluation pipelines scoring large volumes of production calls with low latency.
• Build audio-native metrics validated against human ground truth and LLm evaluators.
• Extend a simulation engine dialing real agents across phone/WebRTC with varied personas and noise.
• Own analytics infra: columnar stores, key-value streams, dashboards, and queries.
• Orchestrate long-running workflows (replays, backfills, load tests) with durability and correctness.
• Operate dashboards, alerts, and root-cause analysis when metrics drift.

Kenntnisse

Backend development

ML systems

Audio processing

TypeScript / Python

Tools

Temporal

PostgreSQL

DynamoDB

ClickHouse

Jobbeschreibung

Role overview

A founding-level engineering position focused on building the evaluation and measurement infrastructure for an AI voice calling platform. The work centers on scoring production calls in near real time, layering audio-native signals (pronunciation, emotion, vocal stress) on top of LLM-based evaluators, and powering a simulation engine that places realistic automated calls. Getting this measurement layer right is treated as foundational to everything else in the product.

Responsibilities

• Design and operate evaluation pipelines that score large volumes of production calls at low latency.
• Build audio-native metrics that go beyond transcripts, validating them against human-labeled ground truth alongside LLM-based evaluators.
• Extend a simulation engine that dials real agents across phone and WebRTC, modeling varied personas, accents, and background noise.
• Own large‑scale analytics infrastructure, including columnar data stores, key‑value streams, and the query layer feeding internal dashboards.
• Orchestrate long‑running, distributed workflows (replays, backfills, load tests) using a durable workflow engine, with strong attention to idempotency and correctness.
• Operate what you build: dashboards, alerts, and root‑cause investigation when production metrics drift.

Requirements

• Four or more years of production experience building backend, ML, or audio systems, ideally in TypeScript or Python.
• Hands‑on depth in at least one of: speech/audio processing, LLM evaluation pipelines, or high‑volume data infrastructure.
• Strong rigor around correctness, with comfort thinking in terms of idempotency, replay safety, and ground‑truth labels rather than only happy paths.
• Willingness to own operations, including dashboards, alerts, and after‑hours investigation.
• Clear written communication, with design documents and post‑mortems people actually want to read.

Nice to have

• Familiarity with the listed stack (TypeScript, Bun, AWS via SST, ClickHouse, DynamoDB, Temporal, Postgres, and WebRTC/telephony integrations).
• Experience validating ML systems against human judgment at scale.