Applied AI Engineer

Synth (YC S21); Bengaluru; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Synth (YC S21) seeks an individual to own the model quality bar for its voice AI platform. This role involves building evals that measure advancements in transcription accuracy and TTS quality, ensuring you understand the impact of every change.

The ideal candidate will have extensive experience in ML engineering, proficiency with Python, and hands-on expertise with LLM and audio data systems. Join a dynamic team at the forefront of voice technology innovation.

Qualifications

• Experience shipping production ML systems.
• Hands-on knowledge of LLM-based agent design.
• Familiarity with audio data processing.

Responsibilities

• Own the model quality evaluation for the voice AI platform.
• Build and maintain eval frameworks and datasets.
• Drive improvements in transcription accuracy and TTS quality.

Skills

ML engineering experience

Strong Python programming

Experience with ML stack (PyTorch, Huggingface)

Hands-on experience with LLM-based agents

Practical experience with ASR/STT

Experience with TTS systems

Comfort with audio data

Job description

About The Role

You’ll own the model quality bar for our voice AI platform, building the evals that tell us if we’re getting better, and driving real, measurable improvements in transcription accuracy and TTS quality. This role sits at the intersection of applied ML, audio, and rigorous experimentation: if you ship a change, you’ll know exactly what it bought us.

What You’ll Do

• Build and maintain the eval framework that scores voice agent quality end‑to‑end transcription, response quality, TTS, and full‑conversation outcomes
• Design voice agent behavior: system prompts, tool use, conversation flow, error recovery, and guardrails for real‑time interactions
• Drive transcription accuracy improvements across STT providers and configurations (Deepgram, Whisper, AssemblyAI, Nvidia, etc.)
• Drive TTS quality improvements voice selection, latency vs. fidelity tradeoffs, prosody, edge cases
• Curate and grow our evaluation datasets, including hard‑case mining from production traffic
• Run rigorous A/B experiments and report results that the team can actually act on
• Partner with backend engineers to wire eval signals into CI so regressions get caught before they ship

Must‑haves

• ML engineering experience shipping production systems
• Strong Python and a working ML stack (PyTorch, Huggingface, pandas, scikit‑learn)
• Hands‑on experience designing LLM‑based agents: prompting, tool/function calling, multi‑turn state, structured outputs
• Hands‑on experience building evals or eval frameworks for ML, LLM, or voice systems. Built LLM‑as‑judge eval pipelines and know their failure modes
• Practical experience with ASR/STT comparing providers, fine‑tuning, or running open models like Whisper
• Practical experience with TTS systems (ElevenLabs or open models)
• Comfortable working with audio data: sample rates, codecs, noise, alignment

Nice‑to‑haves

• Designed voice agents specifically handled barge‑in, interruption recovery, disfluencies, and natural turn‑taking at the prompt/behavior layer
• Experience with diarization, VAD, or endpointing models
• Audio dataset curation, labeling, or annotation pipelines
• Trained or fine‑tuned ASR or TTS models from scratch or on domain audio
• Experience with active learning or data‑flywheel patterns over production traffic
• Open‑source contributions to AI/ML frameworks
• Familiarity with cost/latency tradeoffs across model providers for real‑time voice