Linguist II: AI & LLM Data & Evaluation (Remote)

Spectraforce Technologies; Raleigh (NC); On-site; USD 70,000 - 100,000; Full time

Job summary

Spectraforce Technologies is seeking a Linguist II to contribute to AI-powered language components, including LLMs and voice-enabled tech. You will analyze linguistic data, assist with data collection, annotation, and localization for model training and evaluation.

The role requires strong data analysis skills, programming familiarity, and experience with multilingual speech/text data. Remote work with PST time preference is offered.

Qualifications

• Native English fluency and at least one other language.
• Knowledge of syntax, semantics, pragmatics, sociolinguistics, corpus linguistics, and other areas of linguistics.
• Familiarity with Large Language Models (LLMs), their applications and data practices (training data, evaluation, prompting, fine-tuning).
• Exposure to LLM evaluation methodologies (human evaluation, automated metrics, adversarial testing).
• Experience working with semantic ontologies, taxonomies, or intent/slot frameworks.
• Proficiency using AI Agents/Chatbots.
• Experience with database queries and data analysis processes (SQL, Python, spreadsheets, R, Unix, or others).
• Experience working with speech and text data in multiple languages.
• Comfortable working in a fast-paced, highly collaborative environment with evolving priorities.

Responsibilities

• Apply linguistic expertise in syntax, semantics, pragmatics, and sociolinguistics to support LLM and generative AI systems.
• Collaborate with linguists, data operations teams, and ML engineers on data collection, curation, annotation, and localization efforts for model training and fine-tuning.
• Contribute to the development and maintenance of annotation schemas and guidelines for LLM training data (e.g., instruction-tuning, preference labeling, RLHF).
• Evaluate and quality-check datasets used for pre-training, fine-tuning, and alignment of language models.
• Support the development of programmatic methods for generating synthetic annotated data at scale.
• Assist in model evaluation efforts including prompt-based testing, red-teaming, and linguistic error analysis.
• Contribute to AI safety and responsible AI practices through linguistic review of model outputs (e.g., hallucination detection, bias identification, tone and pragmatic appropriateness).
• Participate in experiments to assess data quality, annotation consistency, and downstream model performance.

Skills

Native English fluency

Multilingual data analysis

Linguistics data analysis

Linguistic theory knowledge (syntax/SE

LLMs familiarity

LLM evaluation methodologies

Ontology/taxonomy experience

AI agents / chatbots familiarity

SQL, Python, R, Unix

Speech and text data in multiple langs

Fast-paced collaborative environment

Education

Bachelor's degree in Linguistics, Computational Linguistics, Computer Science, Speech Science, or related field

Tools

Hugging Face

spaCy

NLTK

PyTorch

Job description

Spectraforce Technologies is seeking a Linguist II to contribute to AI-powered language components, including LLMs and voice-enabled tech. You will analyze linguistic data, assist with data collection, annotation, and localization for model training and evaluation.

The role requires strong data analysis skills, programming familiarity, and experience with multilingual speech/text data. Remote work with PST time preference is offered.