Remote AI QA Architect for LLMs & Azure

Edvwerke; Indiana (PA); Remote; USD 90,000 - 120,000; Full time

Benefits offered by this job

Competitive salary with performance-based bonuses

Opportunities for professional development

Dynamic and collaborative work environment

Job summary

Edvwerke is searching for a QA Consultant to deploy and maintain solutions using advanced Generative AI techniques and RAG architecture. This role requires significant experience in Azure Cloud, particularly deploying and managing models in Azure AI Studio.

The successful candidate will also need to write complex Python scripts and define evaluation metrics for LLMs. The position offers a competitive salary, performance-based bonuses, and opportunities for professional development in a dynamic, remote work environment.

Qualifications

• 1–2 years of hands‑on experience in Generative AI and RAG architecture.
• 5+ years of experience with Azure Cloud and deployment in Azure AI Studio.
• 7+ years of Python coding experience with complex script writing.
• 8+ years in QA & Automation with understanding of LLM metrics.
• 5+ years of DataOps / CI/CD experience using Azure DevOps.

Responsibilities

• Deploy and maintain AI solutions using RAG architecture.
• Implement and manage AI models in Azure AI Studio.
• Write complex Python scripts for data processing.
• Define and execute evaluation metrics for LLMs.
• Build Azure DevOps pipelines for automation.

Skills

Generative AI

Azure Cloud

Python Coding

QA & Automation

DataOps / CI/CD

Education

Technical degree (CS, Engineering, Math)

Tools

Azure AI Studio

Pandas

PyTest

Azure DevOps

Job description

Edvwerke is searching for a QA Consultant to deploy and maintain solutions using advanced Generative AI techniques and RAG architecture. This role requires significant experience in Azure Cloud, particularly deploying and managing models in Azure AI Studio.

The successful candidate will also need to write complex Python scripts and define evaluation metrics for LLMs. The position offers a competitive salary, performance-based bonuses, and opportunities for professional development in a dynamic, remote work environment.