AI Benchmarking & Performance Engineer

DevRev; Metro Manila; On-site; PHP 1,200,000 - 2,400,000; Full time

Job summary

DevRev is seeking seasoned software engineers to help scale AI benchmarking and evaluation for enterprise SaaS. You’ll build benchmarking infrastructure, design evaluation pipelines, and instrument model-serving systems to measure reasoning, accuracy, and user outcomes.

With training and mentorship, you’ll grow in AI benchmarking, profiling, telemetry, and performance tuning while collaborating across AI and Infra teams to shape how intelligence is measured in SaaS 2.0.

Qualifications

• 3–10+ years of backend software engineering experience.
• Strong Python or equivalent backend language background.
• Hands-on API development and cloud-native architectures (AWS/GCP/Azure).
• Familiarity with metrics, profiling, or observability tools.
• Willingness to learn AI benchmarking and LLM evaluation.
• Curiosity with strong analytical and product intuition.

Responsibilities

• Participate in a structured AI benchmarking training track to develop expertise.
• Build and scale benchmarking infrastructure for evaluating AI systems.
• Design agent evaluation pipelines that measure reasoning, accuracy, and user outcomes.
• Collaborate with AI and Infra teams to instrument model-serving systems.
• Develop data and telemetry pipelines capturing model performance and latency.
• Contribute to DevRev's open benchmarking frameworks and KPIs.

Skills

Backend engineering

Python

APIs

Cloud-native

Observability

AI benchmarking

Analytical thinking

Tools

AWS

GCP

Azure

Kubernetes

Grafana

Prometheus

ELK

Job description

DevRev is seeking seasoned software engineers to help scale AI benchmarking and evaluation for enterprise SaaS. You’ll build benchmarking infrastructure, design evaluation pipelines, and instrument model-serving systems to measure reasoning, accuracy, and user outcomes.

With training and mentorship, you’ll grow in AI benchmarking, profiling, telemetry, and performance tuning while collaborating across AI and Infra teams to shape how intelligence is measured in SaaS 2.0.