Senior Natural Language Corpus Evaluation Specialist

Agile 4UX; Abu Dhabi; Remote; AED 350.000 - 550.000; Vollzeit

Benefits dieser Stelle

100% remote work

Competitive compensation

Professional development

Collaboration with international teams

Zusammenfassung

Agile 4UX is seeking a Senior Natural Language Corpus Evaluation Specialist to lead the assessment and improvement of linguistic datasets used in AI, NLP, and ML projects. The role emphasizes accuracy, representativeness, and quality across multilingual corpora, with an emphasis on rigorous evaluation frameworks.

You will collaborate with NLP engineers and researchers, establish standards, review annotations, and develop repeatable QA procedures in a fully remote environment.

Qualifikationen

• Bachelor's or master's degree in Computational Linguistics, Linguistics, NLP, CS, Data Science, or related field.
• Significant experience in corpus evaluation, NLP dataset quality assurance, or linguistic data management.
• Advanced understanding of linguistic structures, language variation, and text analysis methodologies.

Aufgaben

• Lead evaluations of large-scale natural language corpora across monolingual and multilingual datasets.
• Establish corpus quality standards, evaluation frameworks, scoring criteria, and thresholds.
• Collaborate with NLP engineers and researchers to improve dataset quality.
• Develop sampling methodologies and statistical procedures to measure corpus quality.

Ausbildung

Bachelor's or master's degree in Computational Linguistics / NLP / CS / Data Science

Tools

Python

NLP libraries

Jobbeschreibung

Job Title: Senior Natural Language Corpus Evaluation Specialist

Location: United Arab Emirates (Remote)

Employment Type: Full-Time

Work Arrangement: 100% Remote — Work From Home

Level: Senior

About the Role

We are looking for a Senior Natural Language Corpus Evaluation Specialist to lead the assessment, validation, and continuous improvement of linguistic datasets used in artificial intelligence, natural language processing (NLP), and machine learning applications. This role focuses on ensuring that language corpora are accurate, representative, diverse, consistent, and suitable for research and model development.

The successful candidate will bring advanced knowledge of corpus linguistics, language data evaluation, annotation quality, and statistical assessment methodologies.

Key Responsibilities

• Lead comprehensive evaluations of large-scale natural language corpora across monolingual, multilingual, and domain-specific datasets.
• Establish corpus quality standards, evaluation frameworks, scoring criteria, and acceptance thresholds.
• Assess linguistic accuracy, grammatical consistency, semantic relevance, completeness, and dataset representativeness.
• Review text normalization, tokenization, sentence segmentation, metadata, and annotation quality.
• Identify duplicate content, corrupted records, encoding issues, irrelevant material, and inconsistent data structures.
• Evaluate linguistic annotations, including named entities, parts of speech, syntactic relationships, and semantic classifications.
• Analyze corpus composition across languages, dialects, domains, registers, and text sources.
• Detect potential sampling bias, demographic imbalance, linguistic underrepresentation, and data contamination.
• Develop sampling methodologies and statistical procedures to measure corpus quality reliably.
• Assess inter-annotator agreement and investigate inconsistencies in human-generated labels.
• Collaborate with NLP engineers, computational linguists, data scientists, and research teams to improve dataset quality.
• Review automated corpus processing pipelines and recommend improvements to data cleaning and validation workflows.
• Develop repeatable quality assurance procedures and automated evaluation checks.
• Prepare technical reports, evaluation dashboards, and recommendations for stakeholders.
• Maintain documentation covering corpus versions, evaluation results, quality metrics, and corrective actions.
• Provide technical guidance and mentorship to junior corpus evaluators.

Qualifications and Experience

• Bachelor's or master's degree in Computational Linguistics, Linguistics, Natural Language Processing, Computer Science, Data Science, or a related field.
• Significant experience in corpus evaluation, computational linguistics, NLP dataset quality assurance, or linguistic data management.
• Advanced understanding of linguistic structures, language variation, and text analysis methodologies.
• Demonstrated experience assessing large-scale text corpora and multilingual language datasets.
• Knowledge of linguistic annotation frameworks, corpus metadata standards, and dataset validation methods.
• Proficiency in Python and relevant NLP or data processing libraries.
• Experience with corpus analysis tools, text analytics platforms, and automated quality assessment workflows.
• Understanding of tokenization, lemmatization, parsing, named entity recognition, and text normalization.
• Familiarity with statistical sampling, evaluation metrics, and inter-annotator agreement techniques.
• Ability to identify linguistic bias, duplication, inconsistencies, and data quality risks.
• Strong analytical, problem-solving, and technical reporting skills.
• Excellent written communication and stakeholder collaboration abilities.
• Ability to work independently and manage multiple evaluation projects in a fully remote environment.

What We Offer

• 100% remote working arrangement from within the applicable employment jurisdiction.
• Competitive compensation based on experience and technical expertise.
• Opportunities to contribute to advanced AI, NLP, and multilingual language technology initiatives.
• Continuous professional development and exposure to emerging evaluation methodologies.
• Collaboration with international teams across linguistics, data science, and artificial intelligence.