AI Quality & Certification Engineer

Apex Systems; Merrifield (VA); Hybrid; USD 68,880 - 90,921; Part time

Benefits offered by this job

Comprehensive benefits package

401K with company match

Employee stock purchase program (ESPP)

Job summary

Everforth Apex is seeking an experienced AI Quality Engineer in Merrifield, VA (Hybrid) to establish validation and production readiness for AI-powered solutions. You will lead certification reviews, develop quantitative metrics, and collaborate with DevOps to ensure robust, auditable AI systems.

The role emphasizes handling RAG architectures, agent workflows, and cloud-native deployments on Azure services within an enterprise environment.

Qualifications

• Bachelor’s degree in a related technical field is required.
• 5+ years of software quality engineering or ML engineering experience.
• 2+ years of experience with Generative AI, LLMs, or AI agents.

Responsibilities

• Define and implement AI certification, validation, and production readiness standards.
• Develop evaluation frameworks for accuracy, relevance, groundedness, and user satisfaction.
• Design and maintain AI validation datasets and regression suites using LangSmith and Azure Foundry.

Skills

Azure AI Foundry

Azure OpenAI

LangChain

LangGraph

LangSmith

AI agent frameworks

RAG architectures

Azure services

DevSecOps

CI/CD

Education

Bachelor’s Degree in Computer Science

Tools

Azure DevOps

GitHub

LangChain

LangGraph

LangSmith

Azure AI Foundry

Azure OpenAI

Job description

Everforth Apex is seeking an experienced AI Quality Engineer in Merrifield, VA (Hybrid) to establish validation and production readiness for AI-powered solutions. You will lead certification reviews, develop quantitative metrics, and collaborate with DevOps to ensure robust, auditable AI systems.

The role emphasizes handling RAG architectures, agent workflows, and cloud-native deployments on Azure services within an enterprise environment.