Senior AI Agent Engineer — Production-Grade Automation

Falcon Labs; United States; Hybrid; USD 150,000 - 210,000; Full time

Job summary

Falcon Labs seeks a senior engineer to build and run LLM agents, ensuring reliability and measurable improvement through experiments. You will own the agent platform, from runtimes to deployment and CI, and collaborate with AI Enablement to automate real tasks.

You’ll work with tracing, observability, and regression tests, write SQL to prove impact, and expand backend work across TypeScript or Python on AWS. This is a hands-on senior role shaping production AI.

Qualifications

• Must have built and run LLM agents in production with tool use, multi-step workflows, context management, cost and latency budgets.

Responsibilities

• Build and run the platform our agents live on: isolated runtimes, access control, cost limits, CI deployment.
• Instrument every agent with tracing and replay capability across prompts, tool calls, state and results.
• Analyze trajectories to identify errors and improve prompts, context, tools, models, or data.
• Create evals with gold datasets, step-level checks and regression suites that run in CI.
• Translate failures and feedback into new eval cases and monitor quality, latency and cost over time.

Skills

LLM agents

agent evaluation

tracing/observability

backend engineering

SQL writing

Tools

OpenTelemetry

Langfuse

LangSmith

Braintrust

Arize

Job description

Falcon Labs seeks a senior engineer to build and run LLM agents, ensuring reliability and measurable improvement through experiments. You will own the agent platform, from runtimes to deployment and CI, and collaborate with AI Enablement to automate real tasks.

You’ll work with tracing, observability, and regression tests, write SQL to prove impact, and expand backend work across TypeScript or Python on AWS. This is a hands-on senior role shaping production AI.