Remote AI Evaluation Engineer – CI/CD & LLM Testing

Jobgether; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Fully remote

Flexible schedule

Paid training and professional devlop.

Performance-based annual pay increases

negotiable hourly rate

Job summary

Jobgether is hiring to design and build evaluation infrastructure for production-grade AI agents in a fully remote setup. You will develop automated testing systems for complex reasoning and multi-turn conversations, and own the evaluation architecture from ground up.

The role emphasizes Python/TypeScript proficiency, experience with LLM evaluation pipelines, and seamless collaboration across time zones, especially US Central Time. Competitive, remote-friendly compensation offered.

Qualifications

• Experience building LLM evaluation pipelines and integrating tests into CI/CD.
• Excellent English and ability to overlap with US Central Time business hours.
• Fully remote work with flexible schedule.

Responsibilities

• Build CI/CD testing pipelines to validate AI agent behavior.
• Create statistical evaluation frameworks for non-deterministic AI behavior.
• Own the evaluation architecture from ground up in a fully remote environment.

Skills

English

CI/CD testing

LLM evaluation pipelines

English language proficiency

Remote collaboration

Tools

Python

TypeScript

Job description

Jobgether is hiring to design and build evaluation infrastructure for production-grade AI agents in a fully remote setup. You will develop automated testing systems for complex reasoning and multi-turn conversations, and own the evaluation architecture from ground up.

The role emphasizes Python/TypeScript proficiency, experience with LLM evaluation pipelines, and seamless collaboration across time zones, especially US Central Time. Competitive, remote-friendly compensation offered.