AI Evaluation Platform Engineer (Remote US)

Modulus

United States

Remote

USD 140,000 - 220,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health insurance
Paid time off

Job summary

Modulus in the United States is seeking a talented engineer to build internal software that evaluates AI assistants and language models at scale. You will develop services to query models, preserve context for reproducibility, and enable researchers to compare results across models and versions.

The role is full-time and remote within the United States, with the option to work from Scottsdale, AZ. The position offers an annual salary range of $140,000–$220,000, health insurance, and paid time

Qualifications

  • Production software experience in TypeScript or Python.
  • Experience with data pipelines, APIs, and PostgreSQL.
  • Familiarity with language model APIs and nondeterministic outputs.
  • Care for correctness, testing, and reproducibility.

Responsibilities

  • Build services that query AI assistants and language models through APIs and other interfaces.
  • Create data pipelines that store outputs with enough context to reproduce each run.
  • Develop tools for comparing answers across models, versions, and time.
  • Build internal interfaces for running, reviewing, and annotating evaluations.
  • Keep tools reliable, tested, secure, and well documented.

Skills

TypeScript
Python
Data pipelines
APIs
Language Models

Tools

PostgreSQL

Job description

Modulus in the United States is seeking a talented engineer to build internal software that evaluates AI assistants and language models at scale. You will develop services to query models, preserve context for reproducibility, and enable researchers to compare results across models and versions.

The role is full-time and remote within the United States, with the option to work from Scottsdale, AZ. The position offers an annual salary range of $140,000–$220,000, health insurance, and paid time

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Software Engineer, Evaluation Tooling
Software Engineer, Evaluation Tooling

Modulus • United States

Remote
USD 140,000 - 220,000
Health insurance
Paid time off
AI Language Model Evaluator (Remote, Entry)
AI Language Model Evaluator (Remote, Entry)

Meridial • United States

Remote
USD 23,000 - 32,000
AI Evaluation Engineer
AI Evaluation Engineer

DeepRec.ai • Denver (CO)

Remote
USD 162,000 - 198,000
Software Engineer, AI Research & Evaluation
Software Engineer, AI Research & Evaluation

Nearskill | AI & Tech • United States

Remote
USD 106,000 - 184,000
Senior AI Evaluation Infra Engineer (Remote)
Senior AI Evaluation Infra Engineer (Remote)

Turing • United States

Remote
USD 170,000 - 200,000
Equity
Senior Software Engineer, LLM Code Evaluation (US-Based)
Senior Software Engineer, LLM Code Evaluation (US-Based)

OpenTrain AI, Inc. • United States

Remote
USD 83,000 - 165,000
English AI Evaluation Specialist (Remote)
English AI Evaluation Specialist (Remote)

Anyone AI • United States

Remote
USD 9,600 - 17,000
Applied AI Engineer — Remote, Equity, Prompt & Evaluation
Applied AI Engineer — Remote, Equity, Prompt & Evaluation

tend • United States

Remote
USD 120,000 - 180,000
Equity 0.8%–1.2%, vesting over four?ny
Remote work across US/Europe
Senior AI/ML Engineer (Remote) — Expert Model Auditor
Senior AI/ML Engineer (Remote) — Expert Model Auditor

prolificacademicltd • United States

Remote
USD 91,000 - 129,000
Hourly pay for tasks
Fully remote with flexible hours
Skills assessment gates entry
AI Engineer/ML Engineer - Senior Developers - AI Training - Columbus, US
AI Engineer/ML Engineer - Senior Developers - AI Training - Columbus, US

prolificacademicltd • United States

Remote
USD 91,000 - 129,000
Remote work
Flexible scheduling
Competitive hourly pay
+1