Applied AI Engineer: Build Reliable Production Agents

Socket.dev; New York (NY); On-site; USD 140,000 - 210,000; Full time

Job summary

Clay's team builds AI-powered agents that research, enrich, and act for users. You will collaborate with product and engineers to ensure agents are reliable, steerable, and trusted in production.

You’ll design and iterate on agent behavior, build the core harness, improve prompts and context, and establish guardrails and evaluations to measure success and catch regressions. This is a high-impact role in NYC.

Qualifications

• Experience building or shipping production systems with LLMs or agents.
• Strong backend fundamentals—APIs, databases, distributed systems.
• Experience with model or agent evaluation: designing evals, measuring regressions.
• A systems-and-outcomes mindset — you care about whether the product actually works for users, not just model metrics.

Responsibilities

• Design and iterate on agent behavior across real GTM workflows.
• Map manual workflows to agent-driven flows that perform as well as humans.
• Build and run evals to measure task completion and detect regressions.
• Analyze production failures and improve robustness.
• Collaborate with product to move agent flows from prototype to general availability.

Skills

AI systems

Back-end fundamentals

Model evaluation

Systems mindset

Debugging production failures

Tools

React

TypeScript

Python

AWS

Job description

Clay's team builds AI-powered agents that research, enrich, and act for users. You will collaborate with product and engineers to ensure agents are reliable, steerable, and trusted in production.

You’ll design and iterate on agent behavior, build the core harness, improve prompts and context, and establish guardrails and evaluations to measure success and catch regressions. This is a high-impact role in NYC.