Senior AI Developer

ClickUp

Oregon

Híbrido

USD 150 000 - 190 000

Tempo integral

14 dias+

Uma candidatura completa num minuto

Currículo e carta de apresentação personalizados, prontos a enviar.

Ultrapassa os filtros ATS

Resumo da oferta

ClickUp is seeking a Senior AI Engineer focused on AI evaluations to join our engineering team in Oregon. You will design, build, and own ClickUp's AI evaluation system, including golden datasets, LLM-as-judge, and automated scorers.

You will lead model benchmarking, routing decisions, and lifecycle management of AI models in production. You will collaborate across product, frontend, and data teams to deliver reliable, user-centric AI features while advancing privacy and security standards.

Qualificações

  • 5+ years of software engineering experience, including 2+ years building production LLM or ML systems.
  • Hands-on experience building LLM evaluation systems in production with golden datasets and offline/online evals.
  • Strong experiment design and statistical rigor, including A/B testing.
  • Proven track record applying LLMs to real-world product features.
  • Experience integrating multiple LLMs with benchmarking, routing, and cost/latency tradeoffs.
  • Deep expertise in backend engineering, distributed systems, and cloud-native tech.

Responsabilidades

  • Design, build, and own ClickUp's AI evaluation system including datasets and evaluators.
  • Develop the core agent harness and AI features aligned with product goals.
  • Create self-improvement loops where agents modify other agents or prompts.
  • Benchmark models across providers, manage routing, and optimize for quality and cost.
  • Architect scalable AI platform services for deployment and lifecycle management of LLMs.
  • Build robust APIs and backend systems integrating AI features into ClickUp.
  • Develop infrastructure for model serving, monitoring, and evaluation.
  • Lead privacy, security, and regulatory compliance in AI workflows.
  • Collaborate with product, frontend, and data teams to deliver reliable AI experiences.

Conhecimentos

LLM evaluation
Backend engineering
Distributed systems
Python
Go/TypeScript
Cloud platforms
Experiment design
Collaboration
Kubernetes
Docker
AWS/GCP/Azure

Ferramentas

Kubernetes
Docker
AWS/GCP/Azure

Descrição da oferta de emprego

At ClickUp, we're building the future of work: the first truly converged AI workspace unifying tasks, docs, chat, calendar, and enterprise search, all supercharged by context-driven AI. We are an AI-native company. Every team member is expected to leverage AI daily, and we evaluate AI fluency as part of our hiring process. Join us and help redefine what's possible.

We're looking for a highly skilled and experienced Senior AI Engineer to join our ClickUp Engineering team, with a focus on AI evaluations.

Key Responsibilities
  • Design, build, and own ClickUp's AI evaluation system, including golden datasets, LLM-as-judge and automated scorers, regression detection, and quality tracking across AI features.
  • Build and evolve the core agent harness, and the AI that performs well on those evals, partnering closely with product and engineering to define what "great" looks like.
  • Build self-improvement loops where agents modify other agents or prompts, rerun evals, and iterate automatically.
  • Benchmark new models as they're released and own model selection, routing, and fallback strategies across providers (e.g., OpenAI, Anthropic, Google, open-weight), balancing quality, latency, and cost.
  • Architect, design, and implement scalable AI platform services that support the deployment, orchestration, and lifecycle management of LLMs and other AI models.
  • Build and maintain robust APIs and backend systems that integrate AI-powered features into ClickUp's core platform.
  • Develop infrastructure for model serving, monitoring, logging, and automated evaluation to ensure high reliability and performance of AI services in production.
  • Drive best practices in AI privacy, security, and compliance, including data anonymization, secure data handling, and regulatory adherence.
  • Collaborate cross-functionally with product, frontend, and data teams to deliver seamless, reliable, and user-centric AI experiences.
Qualifications
  • 5+ years of software engineering experience, including 2+ years building production LLM or ML systems.
  • Hands-on experience building LLM evaluation systems in production (golden datasets, LLM-as-judge, judge calibration, offline and online evals), tied to real product quality.
  • Strong experiment design and statistical rigor, including A/B testing, offline vs. online metrics, and separating real regressions from noisy or variable results.
  • Proven track record of applying LLMs and AI models to real-world, customer-facing product features.
  • Proven experience integrating and managing multiple LLMs, including benchmarking, routing, and cost/latency/quality tradeoffs.
  • Deep expertise in backend engineering, distributed systems, and cloud-native technologies (e.g., Kubernetes, Docker, AWS/GCP/Azure).
  • Strong programming skills in Python (preferred), Go, TypeScript, or similar languages used for backend and AI platform development.
  • Proficiency in orchestration frameworks and workflow engines (e.g., LangGraph, Airflow, Ray, or similar).
  • Familiarity with search and retrieval technologies and their integration into AI-driven applications.
  • Excellent collaboration and communication skills, with a track record of working effectively in cross-functional teams.
  • Passion for staying at the forefront of AI and applying new technologies to solve real-world problems at scale.
Nice to Have
  • Background in data science, applied science, or machine learning engineering.
  • Experience with eval tooling (e.g., Braintrust, LangSmith, Weave, Arize, or in-house frameworks).
  • Experience with model routing, AI gateways, or inference optimization.

Equal Opportunity Employer

ClickUp is an Equal Opportunity Employer, and qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, or national origin.

Privacy Notice

ClickUp collects and processes personal data in accordance with applicable data protection laws. You can find further details by viewing our Global Candidate Privacy Notice.

If you are a Philippine Job Applicant, please also see our Philippine Data Privacy Notice.

Visa Sponsorship

Please note we are unable to sponsor or take over sponsorship of an employment visa for roles outside of engineering and product at this time. Sponsorship for engineering and product roles is not guaranteed, but is instead based on the business needs for that specific role at that time. Please reach out to the recruiter with any questions.

AI Processing Notice

ClickUp may use artificial intelligence and machine learning technologies to help review and screen candidates' employment applications against role-related criteria. These tools support, but do not replace, human decision-making. If you have questions or need an accommodation in the recruitment process, please contact us at .

Obtém a tua avaliação gratuita e confidencial do currículo.

ou arrasta e larga o ficheiro aqui.

Similar jobs

Ofertas semelhantes que vale a pena comparar

Senior AI Developer
Senior AI Developer

Showcify, Inc. • EUA

Teletrabalho
USD 180 000 - 260 000
Senior AI Developer
Senior AI Developer

ClickUp • EUA

Teletrabalho
USD 140 000 - 210 000
Senior AI Developer
Senior AI Developer

Devconnectplatform • EUA

Teletrabalho
USD 200 000 - 250 000
Senior AI Developer
Senior AI Developer

Learndevs • EUA

Teletrabalho
USD 200 000 - 250 000
Senior AI Developer
Senior AI Developer

Royper Royco LLC • EUA

Teletrabalho
USD 150 000 - 210 000
Health insurance
Staff AI Engineer - Multi-Agent Frameworks
Staff AI Engineer - Multi-Agent Frameworks

ClickUp • EUA

Presencial
USD 120 000 - 160 000
Manager, Software Engineering - Developer Experience
Manager, Software Engineering - Developer Experience

ClickUp • EUA

Teletrabalho
USD 140 000 - 230 000
Remote work
Global team
Manager, Engineering - Product
Manager, Engineering - Product

ClickUp • EUA

Híbrido
USD 140 000 - 190 000
Senior Manager, Software Engineering - Infrastructure
Senior Manager, Software Engineering - Infrastructure

ClickUp • EUA

Teletrabalho
USD 180 000 - 260 000
Senior Software Engineer, Internally Deployed Products
Senior Software Engineer, Internally Deployed Products

ClickUp • EUA

Presencial
USD 120 000 - 160 000