AI Agent Engineer

David Joseph & Company; New York (NY); On-site; USD 200,000 - 300,000; Full time

Benefits offered by this job

Equity

In-person in Manhattan

Early-stage startup environment

Job summary

David Joseph & Company in Manhattan, NY, on-site, seeks a hands-on engineer to build and own the voice AI infrastructure end-to-end. You will design real-time voice agent systems, optimize performance, and craft robust prompting strategies for millions of consumer interactions.

The role requires strong Node.js, AWS, SQL experience, and a business-minded approach to AI outcomes. Equity is offered in a seed-stage startup environment.

Qualifications

• 2+ years of experience in voice AI, conversational AI, prompt engineering, speech systems, or related workflows.
• 3+ years of experience in platform, infrastructure, or backend engineering.
• Strong experience with Node.js, AWS, and SQL.
• Experience building and shipping production systems in a startup or fast-paced environment.
• Experience with voice orchestration platforms or agent frameworks such as LiveKit or equivalent.
• Strong prompt writing instincts and ability to design clear conversational flows for real-world edge cases.
• Strong analytical skills and ability to tie work to business outcomes and ROI.

Responsibilities

• Build the voice AI infrastructure on LiveKit from the ground up.
• Design the architecture of real-time voice agent systems: orchestration, state management, latency optimization, prompting, reliability.
• Own end-to-end prompting strategy: conversation flows, tone, compliance-safe phrasing, guardrails, edge-case handling.
• Build and maintain testing frameworks: A/B testing, regression testing, conversation QA, benchmarking, KPI reporting.
• Expand AI prompting to cover more real-world scenarios.
• Improve containment, conversion, liquidation rates, and customer satisfaction through data-driven iterations.
• Optimize ASR, STT, TTS, turn-taking, interruption handling, and state transitions for voice quality.
• Build scalable systems and tooling for millions of interactions.
• Collaborate with product, operations, and leadership to align technical work with business goals.

Skills

Node.js

AWS

SQL

Voice AI

Prompt engineering

Distributed systems

Tools

LiveKit

SIP/telephony

Real-time audio pipelines

Job description

Manhattan, NY · On-site · Full-time
Compensation: $200,000–$300,000 + 0.25%–1% equity

About the Company

A venture-backed, seed-stage startup using generative AI to automate consumer debt collection, a $35B US market. Its AI agents handle the full late-stage recovery process and already outperform human collectors at a fraction of the cost, running millions of calls per month with strong early revenue traction. Founded 2024 · Seed-stage (1–10 people) · Industry: AI Tools.

The Role

Build and own the company's voice AI agent infrastructure end-to-end on LiveKit — the orchestration layer, the prompting strategy across millions of consumer interactions, and continuous performance improvement through rigorous testing and iteration. This role suits either a deep voice AI engineer or a strong, analytical, business-minded infrastructure engineer.

What you'll be doing

• Build the voice AI infrastructure on LiveKit from the ground up
• Design and own the architecture of real-time voice agent systems: orchestration, state management, latency optimization, prompting, and reliability
• Own the end-to-end prompting strategy: conversation flows, tone, compliance-safe phrasing, guardrails, and edge-case handling
• Build and maintain testing and evaluation frameworks: A/B testing, regression testing, conversation QA, benchmarking, and KPI reporting
• Continuously expand AI prompting to cover more real-world scenarios
• Iterate on performance data and call outcomes to improve containment, conversion, liquidation rates, and customer satisfaction
• Optimize ASR, STT, TTS, turn-taking, interruption handling, and state transitions to improve voice quality
• Build scalable systems and tooling to support millions of voice interactions
• Work closely with product, operations, and leadership to translate business goals into high-performing voice agent behavior

Tech stack: LiveKit, Node.js, AWS, SQL; ASR/STT/TTS, real-time audio pipelines, SIP/telephony

Requirements

• 2+ years of experience in voice AI, conversational AI, prompt engineering, speech systems, or related agentic workflows
• 3+ years of experience in platform, infrastructure, or backend engineering
• Strong experience with Node.js, AWS, and SQL
• Experience building and shipping production systems in a startup or similarly fast-paced environment
• Experience with voice orchestration platforms or agent frameworks such as LiveKit or equivalent
• Strong prompt writing instincts and ability to design clear, effective, and resilient conversational flows for real-world edge cases
• Strong analytical skills and comfort tying technical work directly to business outcomes and ROI

Green Flags

• Voice AI experience in production, even from a no-name company or non-prestigious background.
• Has built AI infrastructure or agentic systems at a startup or AI company, understands what production-grade means at real scale
• LiveKit experience or direct experience with voice orchestration frameworks
• Business-minded, can connect technical decisions to liquidation rates, containment rates, and conversion metrics, not just engineering metrics
• Strong prompt engineering instincts, has designed conversational flows that handle edge cases gracefully under compliance constraints
• Infrastructure depth in Node.js, AWS, and SQL in a startup-paced environment where they owned what they built
• Experience with telephony infrastructure such as SIP trunking is a strong additional signal

Red Flags

• Pure academic or research background with no production deployment experience
• No analytical instincts, builds technically but cannot connect the work to business outcomes and ROI
• English communication skills not at native or near-native level. This is a hard filter due to the prompt engineering and compliance-sensitive consumer communication component
• Not willing to be in-person in Manhattan 4 to 5 days per week
• No experience with real-time systems or low-latency infrastructure

Why Join

• Build the voice AI infrastructure in-house from scratch and own the full stack end-to-end
• Work at real production scale — millions of consumer interactions per month
• Early-stage team with strong traction and clear business impact from day one