Senior Software Engineer: LLM Evaluation & Repo Validation

Turing

United States

Remote

USD 55,000 - 83,000

Part time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Fully remote
Cutting-edge AI projects

Job summary

Aitrainergigs seeks a Senior Software Engineer to lead LLM evaluation and repository validation efforts. This role focuses on high-quality public GitHub projects, automation of environments, issue triage, and assessing test coverage and bug-fixing scenarios.

You will collaborate with researchers, potentially lead junior engineers, and help design tasks based on repositories with 500+ stars. This fully remote contractor role requires about 20 hours per week and starts next week.

Qualifications

  • Experience with one or more of the listed languages and strong navigation of large codebases.
  • Ability to run, test, and modify real-world projects locally.
  • Familiarity with public GitHub repos and issue triaging a plus.

Responsibilities

  • Analyze and triage GitHub issues across trending open-source libraries.
  • Set up and configure code repositories, including Dockerization and environment setup.
  • Evaluate unit test coverage and quality.
  • Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
  • Collaborate with researchers to identify repositories and issues challenging for LLMs.
  • Opportunity to lead a team of junior engineers on projects.

Skills

Python
JavaScript
Java
Go
Rust
C/C++
C#
Ruby

Tools

Git
Docker
CI/CD

Job description

Aitrainergigs seeks a Senior Software Engineer to lead LLM evaluation and repository validation efforts. This role focuses on high-quality public GitHub projects, automation of environments, issue triage, and assessing test coverage and bug-fixing scenarios.

You will collaborate with researchers, potentially lead junior engineers, and help design tasks based on repositories with 500+ stars. This fully remote contractor role requires about 20 hours per week and starts next week.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Go Engineer for LLM Evaluation & Repo Validation
Senior Go Engineer for LLM Evaluation & Repo Validation

Turing • United States

Remote
USD 83,000 - 138,000
Fully remote
Cutting-edge AI projects
Senior Software Engineer - Remote LLM Eval
Senior Software Engineer - Remote LLM Eval

Turing • United States

Remote
USD 41,000 - 83,000
Fully remote
Remote Senior Ruby Engineer — LLM Evaluation & Validation
Remote Senior Ruby Engineer — LLM Evaluation & Validation

Turing • United States

Remote
USD 83,000 - 124,000
Fully remote environment
Cutting-edge AI projects
Remote LLM Evaluation & Repo Validation Engineer
Remote LLM Evaluation & Repo Validation Engineer

RemoExperts • United States

Remote
USD 69,000 - 138,000
Senior JS Code Reviewer for Open-Source & LLM QA
Senior JS Code Reviewer for Open-Source & LLM QA

Turing Global India • United States

Remote
USD 25,000 - 30,000
Fully remote
Work on cutting-edge AI projects
Senior Software Engineer – LLM Evaluation & Repository Validation
Senior Software Engineer – LLM Evaluation & Repository Validation

Turing • United States

Remote
USD 55,000 - 83,000
Fully remote
Cutting-edge AI projects
Software Engineer – LLM Evaluation & Repository Validation
Software Engineer – LLM Evaluation & Repository Validation

RemoExperts • United States

Remote
USD 69,000 - 138,000
Senior Software Engineer, LLM Code Evaluation (Remote, Part-Time)
Senior Software Engineer, LLM Code Evaluation (Remote, Part-Time)

OpenTrain AI, Inc. • United States

Remote
USD 83,000 - 165,000
Senior Software Engineer - LLM Code Evaluation (Remote)
Senior Software Engineer - LLM Code Evaluation (Remote)

OpenTrain AI, Inc. • United States

Remote
USD 55,000 - 110,000
Senior Software Engineer – Ruby (LLM Evaluation & Repository Validation)
Senior Software Engineer – Ruby (LLM Evaluation & Repository Validation)

Turing • United States

Remote
USD 83,000 - 124,000
Fully remote environment
Cutting-edge AI projects