Get a reply from this employer
Your resume and cover letter, tailored to exactly what they’re hiring for.
Management Solutions International (MSI) seeks a Senior AI Evaluation Engineer to lead evaluation, testing and validation of advanced AI and LLM systems in production contexts. This role emphasizes ensuring accuracy, reliability and safety of AI-powered applications.
You will design evaluation frameworks, benchmarks and automated pipelines, and guide continuous improvement through statistical analysis and controlled experiments. Doha, Qatar-based with senior responsibilities.
Senior AI Evaluation Engineer – LLMs & Agentic AI
Location: Qatar
Experience: 7+ Years
Education: Master's Degree or Higher in Computer Science, Software Engineering or a related Engineering discipline.
About the Role
We are looking for an experienced Senior AI Evaluation Engineer to lead the evaluation, testing, validation and continuous improvement of advanced AI and Large Language Model (LLM) systems.
This is not a conventional Machine Learning Engineer or Generative AI Developer position focused primarily on building and deploying AI applications.
The successful candidate will specialize in determining whether AI systems are actually performing correctly, reliably and consistently in real-world production environments.
You will design and implement evaluation frameworks, benchmark datasets, automated testing pipelines and statistical validation methodologies to measure the quality, accuracy, factual reliability and performance of LLM-powered applications and agentic AI systems.
Key Responsibilities
1. LLM & Generative AI Evaluation
2. Evaluation Frameworks & Benchmarking
3. Statistical Analysis & Experimental Validation