Senior AI Agent Engineer - Build Precise, Multi-Step Agents

FirstIgnite; Vitoria; Presencial; EUR 70.000 - 90.000; Jornada completa

Descripción de la vacante

FirstIgnite is hiring a Senior AI Agent Engineer to build long-running, multi-step agents and the evals measuring their improvements. You will craft tools, wrappers, and evaluations across diverse AI SDKs and APIs, with a strong emphasis on accuracy and traceability.

You will collaborate with product and the full-stack team to turn vague quality goals into measurable datasets, providing clear citations and safe handoffs to humans when needed.

Formación

• 3+ years engineering with LLM or agent systems used by real users.
• Experience evaluating agents, not just models, for multi-step runs.
• Integrated with external APIs and built reliable agent calls.
• Proficient with document pipelines: extraction, normalization, and truth matching.
• Experience with at least one LLM evaluation framework.

Responsabilidades

• Design and ship long-running, multi-step, tool-using agents on various AI SDKs and tooling, including the OpenAI Agents SDK, Anthropic Agent SDK, Vercel AI SDK, LangGraph, MCP, and Temporal Cloud.
• Wrap our APIs and our partners' APIs as tools an agent can call over MCP; translate old systems.
• Get structured data out of long documents and match against existing records; include batch processing.
• Stand up eval suites using frameworks like Promptfoo, Braintrust, LangSmith, DeepEval, LLM-as-judge methods, and custom harnesses to measure tool-use correctness and trajectory quality.
• Every agent produces a draft that a person signs off on; build citations and confidence signals that speed reviews and provide escape paths for the agent.
• Collaborate with product and domain experts to turn vague goals into measurable outcomes, even with tiny or confidential datasets.
• Instrument production traffic, convert real interactions into golden datasets, and run regression tests.
• Compare models and prompts across OpenAI, Anthropic, and open-weight options, assessing latency and quality.
• Bootstrap quality signals for features with no production traffic by generating synthetic documents and test cases.

Conocimientos

LLM/agent systems

Agent evaluation

API integration

Document pipelines

LLM evaluation frameworks

Descripción del empleo

FirstIgnite is hiring a Senior AI Agent Engineer to build long-running, multi-step agents and the evals measuring their improvements. You will craft tools, wrappers, and evaluations across diverse AI SDKs and APIs, with a strong emphasis on accuracy and traceability.

You will collaborate with product and the full-stack team to turn vague quality goals into measurable datasets, providing clear citations and safe handoffs to humans when needed.