Senior AI Platform Engineer - LLM Evaluation & Systems

Royper Royco LLC; United States; Remote; USD 150,000 - 210,000; Full time

Benefits offered by this job

Health insurance

Job summary

ClickUp is seeking a highly skilled Senior AI Engineer to join our Engineering team with a focus on AI evaluations. You will own how we measure and improve the quality of ClickUp's AI: building eval systems, the core agent harness, and evaluation-driven improvements across providers.

You'll work on backend services that scale, ensure reliability and security, and collaborate with product and data teams to deliver user-centric AI experiences across ClickUp's platform.

Qualifications

• 5+ years of software engineering experience, including 2+ years building production LLM or ML systems.
• Hands-on experience building LLM evaluation systems in production with gold datasets and judges.
• Strong design and statistical rigor for A/B testing and metrics.
• Proven track record applying LLMs to product features.
• Experience integrating multiple LLMs with routing and cost/latency tradeoffs.
• Backend and cloud-native expertise with Python, Go, TypeScript; Kubernetes and Docker.

Responsibilities

• Design, build, and own ClickUp's AI evaluation system and datasets.
• Develop core agent harness and AI that performs well on evals.
• Create scalable AI platform services for LLM deployment and lifecycle.
• Build robust APIs and backend systems integrating AI features.
• Implement model serving, monitoring, and automated evaluation.
• Lead privacy, security, and compliance in data handling.

Skills

LLM evaluation

Backend engineering

Python

Go

TypeScript

Kubernetes

Docker

Cloud platforms

LangGraph

Airflow

Ray

AI privacy

Tools

LangGraph

Airflow

Ray

Docker

Kubernetes

AWS/GCP/Azure

Job description

ClickUp is seeking a highly skilled Senior AI Engineer to join our Engineering team with a focus on AI evaluations. You will own how we measure and improve the quality of ClickUp's AI: building eval systems, the core agent harness, and evaluation-driven improvements across providers.

You'll work on backend services that scale, ensure reliability and security, and collaborate with product and data teams to deliver user-centric AI experiences across ClickUp's platform.