Linguist - Speech & Conversational AI

BeSimple AI; San Mateo (CA); On-site; USD 70,000 - 100,000; Full time

Job summary

Besimple AI is building datasets, benchmarks, and training environments to help voice models understand people and interact naturally. We’re seeking a linguist to shape the quality of speech and conversations used to train these models, translating linguistic insight into practical decisions for recordings, annotation, and evaluation.

You’ll define criteria, build rubrics, and guide contributors, reviewers, and engineers to ensure consistency and high-quality data for conversational AI research

Qualifications

• Background in linguistics, speech science, phonetics, phonology OR related field.
• Strong listening and analytical skills to describe features of speech.

Responsibilities

• Evaluate speech and conversational quality in recordings and model outputs.
• Create quality standards and evaluation rubrics for transcripts and annotations.
• Screen and support speech contributors and provide actionable feedback on delivery.
• Create and review linguistic annotations with guidelines and reference examples.
• Collaborate with researchers to refine scenarios, instructions, and rubrics.

Skills

Linguistic analysis

Transcription quality

Conversations evaluation

Feedback communication

Education

Linguistics or related field

Tools

Praat

ELAN

Phonetic tools

Job description

Besimple AI is building datasets, benchmarks, and training environments that help voice models understand people, interact naturally, and complete real tasks.

We’re looking for a linguist to help define and improve the quality of the speech and conversations used to train these models. You’ll bring a trained ear for language, an understanding of how people communicate, and the ability to explain what makes a conversation clear, natural, and appropriate to its context.

Your work will connect linguistic expertise with practical decisions: which recordings are suitable for training, how contributors should be evaluated, what annotators should label, and how we measure the behavior of conversational agents.

What you’ll work on

• Evaluate speech and conversational quality. Assess pronunciation, intelligibility, prosody, pacing, tone, and turn-taking in human recordings and model outputs. Explain why a response sounds natural, awkward, overly scripted, or inappropriate for the situation.
• Build quality standards and evaluation rubrics. Translate customer requirements and feedback into clear criteria, examples, and decision rules. Define what reviewers should accept, flag, or reject, including how expectations differ across conversational roles and tasks.
• Screen and support speech contributors. Help design auditions, evaluate recordings, and give actionable feedback on delivery. Identify contributors who can sustain natural conversations while following a scenario’s requirements.
• Create and review linguistic annotations. Develop guidelines and reference examples for transcripts, speaker turns, speech delivery, and conversational behavior. Review human and model-generated labels, resolve ambiguous cases, and improve consistency across reviewers.
• Improve conversational tasks and model evaluation. Work with researchers to refine scenarios, instructions, and turn‑level rubrics. Identify unclear wording or grading criteria, and distinguish real model failures from problems in the task itself.
• Turn recurring problems into better workflows. Analyze disagreements and rejection patterns, calibrate reviewers, and improve guidance in the crowdsourcing platform we build. Partner with engineers and audio specialists to separate language and delivery issues from recording defects.

We welcome recent graduates with relevant research or substantial projects, as well as linguists with industry experience. Research, annotation guidelines, evaluation studies, and examples of applied linguistic work are all useful evidence.

What we’re looking for

• A background in linguistics, speech science, phonetics, phonology, or a related field, through academic study or substantial applied experience.
• Strong listening and analytical skills, with the ability to describe specific features of speech and explain how they affect meaning, clarity, and naturalness.
• Experience with speech data, linguistic annotation, transcription, language assessment, or conversational analysis.
• The ability to turn nuanced judgments into guidelines that other people can apply consistently.
• Clear written communication and comfort giving specific, constructive feedback.
• Careful judgment about ambiguity, language variation, and the difference between personal preference and a task’s actual requirements.

Especially relevant experience

• Evaluating text-to-speech, speech recognition, speech-to-speech, or conversational AI systems.
• Prosody, pragmatics, discourse analysis, sociolinguistics, or conversation analysis.
• Praat, ELAN, phonetic transcription, or other speech‑analysis and annotation tools.
• Designing annotation schemes, building reference datasets, or measuring agreement between reviewers.
• Speaker auditions, voice coaching, or evaluating spoken performance.
• Working with multilingual datasets or collaborating with linguists across languages.
• Basic scripting or data analysis for investigating annotation quality and recurring errors.

You do not need experience in every area listed above.

Why this role

You’ll help shape the standards behind the data that voice models learn from. Your judgments will influence how we collect conversations, select speakers, evaluate model behavior, and improve the quality of our datasets.

This is an opportunity to apply linguistic expertise directly to the development of conversational AI, with ownership over both the quality of individual examples and the processes that make that quality repeatable.