AI Evaluation Platform Engineer

Jobtailor; California (MO); On-site; USD 120,000 - 160,000; Full time

Job summary

Jobtailor is seeking a software-focused engineer to help build an end-to-end evaluation ecosystem for agentic AI frameworks. You will contribute to infrastructure, deployment, and data pipelines that support scalable AI workloads.

The role requires strong software development experience, experience with Python/C++/TypeScript/Rust/Go, Docker, Linux CLI, REST APIs, and SQL. You will collaborate with global teams to advance AI evaluation tooling.

Qualifications

• 3+ years of experience in a STEM field after a Bachelor's degree.
• Master's degree with 6+ months of related educational or work experience.
• Software development experience.
• Experience with Python, C++, TypeScript, Rust or Go.
• Containerized development with Docker, Linux, and CLI tooling.
• Experience with REST APIs and HTTP integrations.
• SQL (SQLite, PostgreSQL) for data querying.
• Experience with AI agents, LLMs, or modern AI infrastructure.

Responsibilities

• Contribute to building infrastructure and tooling that supports an end-to-end evaluation ecosystem for agentic AI frameworks.
• Build and maintain scalable deployment and orchestration systems for AI workloads.
• Develop data pipelines, dashboards, and observability tooling.
• Integrate and evaluate AI agent frameworks and model-serving systems.
• Design and automate evaluation pipelines for performance, reliability, and regression detection.
• Collaborate with global engineering teams to improve AI evaluation tooling.

Skills

Software development

Python

C++

TypeScript

Rust

Go

Docker

Linux

REST APIs

SQL

AI agents/LLMs

Education

Bachelor's degree in STEM field

Master's degree in STEM field

Tools

Docker

Linux

REST APIs

SQL (SQLite, PostgreSQL)

Job description

Jobtailor is seeking a software-focused engineer to help build an end-to-end evaluation ecosystem for agentic AI frameworks. You will contribute to infrastructure, deployment, and data pipelines that support scalable AI workloads.

The role requires strong software development experience, experience with Python/C++/TypeScript/Rust/Go, Docker, Linux CLI, REST APIs, and SQL. You will collaborate with global teams to advance AI evaluation tooling.