Remote AI Evaluation Engineer (Contract)

Alignerr

United States

Remote

USD 83,000 - 165,000

Part time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Alignerr is seeking a Senior Software Engineer to design and scale evaluation infrastructure that measures AI performance. You will build tools, pipelines, and frameworks that help research teams understand model capabilities, failure modes, and how to improve systems.

The role emphasizes high-impact software engineering in a remote setting, requiring 4+ years of experience, strong Python skills, and cloud/CI/CD familiarity. This is an hourly contract with 20–40 hours per week.

Qualifications

  • 4+ years of professional software engineering experience.
  • Strong Python proficiency with data processing libraries.
  • Experience building scalable production systems and APIs.

Responsibilities

  • Design and build scalable evaluation pipelines and frameworks for AI model performance.
  • Develop automated testing harnesses, scoring systems, and benchmarking tools.
  • Write clean, production-quality code to process, analyze, and visualize evaluation datasets at scale.
  • Create and maintain APIs, dashboards, and internal tools for research teams to run evaluations.
  • Collaborate with AI researchers to translate evaluation methodologies into software.
  • Identify edge cases and reliability issues in AI outputs.
  • Optimize system performance and infrastructure costs.
  • Contribute to architecture and technical direction of the platform.

Skills

Python
Data processing (pandas/NumPy)
Web frameworks (FastAPI/Flask/Django)
Cloud / CI/CD familiarity

Tools

FastAPI
Flask
Django
Git
CI/CD

Job description

Alignerr is seeking a Senior Software Engineer to design and scale evaluation infrastructure that measures AI performance. You will build tools, pipelines, and frameworks that help research teams understand model capabilities, failure modes, and how to improve systems.

The role emphasizes high-impact software engineering in a remote setting, requiring 4+ years of experience, strong Python skills, and cloud/CI/CD familiarity. This is an hourly contract with 20–40 hours per week.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Python Infra Engineer for AI Model Evaluation
Remote Python Infra Engineer for AI Model Evaluation

Alignerr • United States

Remote
USD 83,000 - 165,000
Senior AI Infrastructure Engineer — Remote & High-Impact
Senior AI Infrastructure Engineer — Remote & High-Impact

Alignerr • United States

Remote
USD 96,000 - 165,000
Fully remote
Remote Senior Python Full-Stack Engineer for AI Data Infra
Remote Senior Python Full-Stack Engineer for AI Data Infra

Alignerr • United States

Remote
USD 96,000 - 152,000
Senior Software Engineer – AI Evaluation
Senior Software Engineer – AI Evaluation

Alignerr • United States

Remote
USD 83,000 - 165,000
Remote AI Training Software Engineer (Contract)
Remote AI Training Software Engineer (Contract)

Alignerr Corp. • Austin (TX)

Remote
USD 14,000 - 55,000
Remote Python Engineer for AI Workflows & Data Pipelines
Remote Python Engineer for AI Workflows & Data Pipelines

Alignerr • United States

Remote
USD 83,000 - 124,000
Senior Backend Engineer: AI Tooling & Data Pipelines, Remote
Senior Backend Engineer: AI Tooling & Data Pipelines, Remote

Alignerr Corp. • Seattle (WA)

Remote
USD 96,000 - 165,000
Fully remote
Flexible hours
Senior Backend Engineer, AI Tooling — Remote
Senior Backend Engineer, AI Tooling — Remote

Alignerr Corp. • New York (NY)

Remote
USD 96,000 - 165,000
Remote Backend Python Engineer — AI Code Evaluation
Remote Backend Python Engineer — AI Code Evaluation

Alignerr • United States

Remote
USD 83,000 - 152,000
Remote AI Trainer & Evaluator (Contractor)
Remote AI Trainer & Evaluator (Contractor)

Frontier AI Lab • United States

Remote
USD 83,000 - 165,000