Staff AI Engineer – AI Labs

Jobtailor; Madrid; Presencial; EUR 120.000 - 150.000; Jornada completa

Descripción de la vacante

dLocal is seeking a senior technologist to validate high-value AI and automation technologies, running instrumented LLM experiments, prototypes, and benchmarks to de-risk adoption across the company.

You will design evaluation environments, build tooling, write concise decision memos, mentor engineers, and collaborate with Security, Legal, Compliance and product teams to align on standards and governance.

Formación

• 8+ years of software engineering experience, including senior/Staff-level scope.
• Deep hands-on experience building and evaluating AI/ML systems based on LLMs and tooling.
• Strong fundamentals and ability to rapidly build high-quality experimental systems.
• Experience building agentic or multi-step AI systems with tool use and integrations.
• Strong knowledge of cloud infrastructure, preferably AWS, with secure, cost-conscious workloads.
• Experience with observability, telemetry, testing, and benchmarking of complex systems.
• Ability to design experiments and evaluation plans with hypotheses and metrics.
• Ability to explain technical results to non-specialists.

Responsabilidades

• Validate high-value AI and automation technologies and de-risk adoption across the company.
• Own technology scouting, prototyping, and evaluation for the business.
• Run instrumented spikes and benchmarks on LLMs, agents, and copilots.
• Compare vendor and open-source options across quality, cost, latency, security.
• Deliver decision memos with recommendations to adopt, watch, or avoid.
• Design and maintain evaluation environments with datasets, prompts, and telemetry.
• Build automation and tooling to measure quality, latency, cost, and regressions.
• Mentor engineers on evaluation methods and experimentation design.
• Share knowledge through internal talks and write-ups.

Conocimientos

8+ years software engineering

Senior/Staff-level scope

LLMs evaluation

Influencing without authority

Curiosity

Mentoring

Communication

Herramientas

LLMs

Agentic Systems

Vector Databases

Orchestration Frameworks

Telemetry Tools

Automation and Tooling

AWS AI Suite

Descripción del empleo

• Validate high-value emerging AI and automation technologies and de-risk their adoption across dLocal
• Own technology scouting, prototyping, and evaluation for dLocal
• Run instrumented spikes and benchmarks on LLMs, agentic systems, vector databases, orchestration frameworks, copilots, assistants, and other emerging technologies
• Compare vendor and open-source options across quality, cost, latency, security, and integration complexity
• Deliver decision memos with recommendations to adopt, watch, or avoid
• Design and maintain evaluation environments with datasets, prompts, scenarios, and telemetry
• Build automation and tooling to measure quality, robustness, latency, cost, and regressions
• Build focused prototypes to explore architecture, integration patterns, operational constraints, security boundaries, and failure modes
• Define readiness guidance, patterns, guardrails, limitations, operational considerations, and integration requirements
• Coordinate hand-offs to engineering teams responsible for productionization and support transitions as needed
• Track outcomes of Lab recommendations to improve evaluation methods
• Work with Security, Legal, Compliance, and AI teams on risk assessments and governance recommendations
• Maintain reusable checklists, decision templates, and standards
• Incorporate learnings from external copilots and the AWS AI suite into adoption guidelines
• Partner with AI and domain teams to ensure collaboration and clear boundaries
• Participate in hiring as a technical evaluator and culture champion
• Mentor engineers on evaluation methods, benchmarking, and experimental design
• Share knowledge through internal write-ups, tech talks, meetups, and conferences

Requirements

• 8+ years of software engineering experience, including significant experience operating at senior or Staff-level scope
• Deep hands-on experience building and evaluating systems based on LLMs and modern AI tooling
• Strong software engineering fundamentals and ability to rapidly build high-quality experimental systems
• Experience building agentic or multi-step AI systems involving tool use, orchestration, state, retrieval, or external integrations
• Strong knowledge of cloud infrastructure, preferably AWS, and ability to run experimental workloads securely and cost-consciously
• Experience with observability, telemetry, testing, and benchmarking of complex systems
• Ability to reason about system architecture, reliability, scalability, asynchronous workflows, and distributed components
• Track record of designing experiments or benchmarks that influenced meaningful technical decisions
• Experience constructing evaluation datasets, including task selection, labelling, and holdout discipline
• Working knowledge of LLM-as-judge methods, human evaluation, inter-annotator agreement, and their appropriate use
• Ability to reason about statistical significance on small samples
• Familiarity with regression tracking, telemetry, and versioning
• Ability to turn ambiguous ideas into scoped evaluation plans with hypotheses and metrics
• Comfortable making trade-off calls across quality, latency, cost, and vendor lock-in
• Experience writing concise decision memos
• Ability to explain technical results to non-specialists
• Experience working with platform, product, and operations teams
• Ability to influence without authority and align teams around standards and guardrails
• Curious, experimentation-oriented mindset with disciplined measurement and risk awareness
• Comfortable in a small, high-leverage team without embedded PMs
• Builder attitude favoring reusable tools, templates, and playbooks

Core Competencies

Demonstrates extensive experience in evaluating and adopting AI and automation technologies, with a strong focus on LLMs and cloud infrastructure, particularly AWS. Capable of designing experiments, building prototypes, and collaborating across teams to ensure effective integration and governance.

Highest-signal resume keywords

• LLM Evaluation
• Cloud Infrastructure (AWS)
• Experimental Design
• Benchmarking and Telemetry
• Decision Memo Writing

Hard Skills

• Software Engineering
• AI Tooling
• System Architecture
• Observability
• Statistical Significance Reasoning
• Evaluation Dataset Construction
• Regression Tracking
• Integration Patterns
• Automation and Tooling
• Prototyping

Soft Skills

• Influencing Without Authority
• Curiosity
• Collaboration
• Mentoring
• Communication

Industry Keywords

• AI Governance
• Risk Assessment
• Compliance
• Experimental Workloads
• Vendor Evaluation

Tools & Technologies

• LLMs
• Agentic Systems
• Vector Databases
• Orchestration Frameworks
• Telemetry Tools
• Decision Templates
• Reusable Checklists
• AWS AI Suite