Senior Python AI Evaluation Engineer

AI Trainer Gigs

San Francisco (CA)

Hybrid

USD 55,000 - 110,000

Full time

13 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Turing, based in San Francisco, seeks a Software Engineering evaluator to create cutting-edge datasets for training and benchmarking large language models. You will curate code examples and solutions, focusing on Python across backend and ML infrastructure, with exposure to JS, C/C++, Java, Rust, and Go.

You will evaluate AI-generated code for efficiency and reliability, work with researchers and cross-functional teams, and build verification tools in Python.

Qualifications

  • Several years of software engineering experience (3+ years).
  • Strong Python expertise with production-grade software practices.
  • Experience building and deploying full-stack apps.
  • Deep understanding of software architecture and code quality.
  • Excellent oral and written communication for evaluation rationale.

Responsibilities

  • Curate and evaluate AI-driven coding solutions.
  • Collaborate with researchers and cross-functional teams.
  • Develop verification tools and scripts in Python.
  • Assess performance against industry benchmarks.
  • Hypothesize software engineering steps and assess model capabilities.
  • Design verification mechanisms for coding tasks.

Skills

Python
Full-stack development
Software architecture
Code quality and reviews
Communication skills

Job description

Turing, based in San Francisco, seeks a Software Engineering evaluator to create cutting-edge datasets for training and benchmarking large language models. You will curate code examples and solutions, focusing on Python across backend and ML infrastructure, with exposure to JS, C/C++, Java, Rust, and Go.

You will evaluate AI-generated code for efficiency and reliability, work with researchers and cross-functional teams, and build verification tools in Python.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Python Engineer - AI Data & Model
Senior Python Engineer - AI Data & Model

Turing • United States

Remote
USD 69,000 - 96,000
Senior AI Software Evaluator — Python & ML Infra
Senior AI Software Evaluator — Python & ML Infra

AI Trainer Gigs • San Francisco (CA)

On-site
USD 83,000 - 138,000
Senior Python Engineer — AI Training Data Pipelines
Senior Python Engineer — AI Training Data Pipelines

HumanitApp • San Francisco (CA)

On-site
USD 83,000 - 124,000
AI Data & Code Evaluation Engineer - Python Focus
AI Data & Code Evaluation Engineer - Python Focus

Turing • United States

Remote
USD 83,000 - 131,000
Remote AI Software Evaluator (Python/ML) — Contractor
Remote AI Software Evaluator (Python/ML) — Contractor

Turing • Los Angeles (CA)

Remote
USD 55,000 - 96,000
Remote Python-Powered AI Full-Stack Engineer
Remote Python-Powered AI Full-Stack Engineer

Turing • Pittsburgh

Remote
USD 55,000 - 117,000
Remote AI Software Engineer - Python & ML Infra
Remote AI Software Engineer - Python & ML Infra

Turing • San Francisco (CA)

Remote
USD 83,000 - 165,000
Senior Backend Engineer (Python) for AI Benchmarking
Senior Backend Engineer (Python) for AI Benchmarking

Turing • United States

Remote
USD 83,000 - 131,000
AI-Driven Full-Stack Engineer (Python)
AI-Driven Full-Stack Engineer (Python)

Turing • Pittsburgh

Remote
USD 83,000 - 138,000
Senior Python Engineer for AI Data & LLM Training
Senior Python Engineer for AI Data & LLM Training

HumanitApp • United States

Remote
USD 69,000 - 96,000