AI Benchmarking Engineer

Wipro Limited

San Diego (CA)

On-site

USD 60,000 - 149,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Medical and dental benefits
Disability insurance
Paid time off

Job summary

Wipro Limited in San Diego, CA, is seeking an engineer to build and scale evaluation tooling for AI-powered software development. You will develop evaluation harnesses, automate benchmark runs, and ensure results are reproducible and auditable through CI/CD environments.

The role blends engineering with measurement discipline, demanding strong programming skills (Python/Java/JavaScript), experience with Docker and Git, and familiarity with LLM-based evaluation and agent benchmarking.

Qualifications

  • Strong software engineering background with automation experience.
  • Experience evaluating AI coding agents and validating code changes.
  • Ability to build reproducible evaluation workflows and environments.
  • Familiarity with Git history, branches, and CI/CD pipelines.
  • Knowledge of Docker/containerization and API-driven workflows.
  • Experience with AI/LLM-based evaluation and tool benchmarking.

Responsibilities

  • Build and integrate evaluation harnesses and automation for software development use cases.
  • Create versioned, repeatable evaluation processes with reproducible runs.
  • Validate evaluation approaches against human judgment to ensure correctness.
  • Support benchmarking across quality, productivity, and cost metrics.
  • Analyze results across runs and improve reliability and automation.
  • Document evaluations for technical and leadership audiences.

Skills

Software engineering
Automation tooling
CI/CD
Git
Docker
Python/Java/JavaScript
LLM evaluation
Benchmarking
Troubleshooting
Team collaboration

Tools

Docker
Git
CI/CD tools
Claude Code
Devin
Cursor

Job description

Wipro Limited in San Diego, CA, is seeking an engineer to build and scale evaluation tooling for AI-powered software development. You will develop evaluation harnesses, automate benchmark runs, and ensure results are reproducible and auditable through CI/CD environments.

The role blends engineering with measurement discipline, demanding strong programming skills (Python/Java/JavaScript), experience with Docker and Git, and familiarity with LLM-based evaluation and agent benchmarking.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Evaluation Engineer
AI Evaluation Engineer

Wipro Limited • San Diego (CA)

On-site
USD 60,000 - 149,000
Medical and dental benefits
Disability insurance
Paid time off
AI Systems Benchmark Architect
AI Systems Benchmark Architect

Hewlett Packard Co. • Spring (TX)

Hybrid
USD 131,000 - 205,000
Health insurance
Dental insurance
Vision insurance
+3
AI Benchmarking & Performance Architect
AI Benchmarking & Performance Architect

Coreweave • Sunnyvale (CA)

On-site
USD 206,000 - 333,000
Medical, dental, and vision insurance
Equity awards
401(k) with match
+2
AI Benchmarking & Strategy Lead
AI Benchmarking & Strategy Lead

Artificial Analysis • San Francisco (CA)

On-site
USD 180,000 - 260,000
Equity
Competitive compensation
AI Benchmark Engineer (Remote Contract)
AI Benchmark Engineer (Remote Contract)

LinkedIn • United States

Remote
USD 83,000 - 165,000
AI Evaluation Engineer, Benchmarks & Tooling
AI Evaluation Engineer, Benchmarks & Tooling

Bake AI Inc. • San Mateo (CA)

On-site
USD 180,000 - 240,000
Senior AI Evaluation & Benchmark Engineer (Hybrid)
Senior AI Evaluation & Benchmark Engineer (Hybrid)

Glean • San Francisco (CA)

Hybrid
USD 200,000 - 280,000
Medical, Vision, Dental
Generous time-off
401k plan
+4
AI Benchmarking & Evaluation Scientist
AI Benchmarking & Evaluation Scientist

snorkelai • United States

Remote
USD 120,000 - 180,000
Senior Software Engineer, AI Benchmark Platform
Senior Software Engineer, AI Benchmark Platform

Turing • United States

Remote
USD 140,000 - 210,000
AI Performance & Benchmark Engineer
AI Performance & Benchmark Engineer

Hp Iq • Houston (TX)

On-site
USD 131,000 - 205,000
Health insurance
Vision insurance
Disability insurance
+4