Founding AI Engineer

Vacaturebank; Amsterdam; Hybrid; EUR 194,000 - 220,000; Full time

Job summary

Vacaturebank is seeking a founding engineer in Amsterdam to own evaluation systems, feedback loops, and LLM infrastructure for a pricing intelligence B2B SaaS. You’ll influence the technical direction from day one.

You will build eval harnesses, automate expert reviews, and ship end-to-end LLM features while ensuring data residency and GDPR compliance in a lean startup environment.

Qualifications

• 8+ years of engineering experience with strong, recent production LLM depth.
• Proven track record shipping and owning LLM-powered features end-to-end.
• Hands-on experience building eval harnesses and observability for LLM systems.
• Experience with MCP or building tools for LLM agents; familiarity with LangChain, LlamaIndex, Braintrust, or OpenRouter is a plus.
• Experience with structured data models and typed ontologies (e.g., Pydantic).
• Background operating under data residency, SOC2, and GDPR constraints in pricing/billing/payments.
• Strong communication skills to explain non-deterministic systems to clients and stakeholders.
• Authorized to work in the United States; visa sponsorship is not available.

Responsibilities

• Build eval harnesses and benchmarks using tracked pricing outcomes as ground truth for model quality.
• Systematize and automate expert review workflows that are currently handled manually.
• Develop AI personas that simulate B2B buying committees using usage data and call transcripts.
• Automate persona training pipelines and reduce reliance on manual processes.
• Own LLM routing across multiple providers with explicit cost, latency, and quality tradeoffs.
• Maintain infrastructure and data residency boundaries (e.g., keeping EU model calls within the EU).
• Extend the MCP server used by LLM agents so features are agent-driven, not just UI-rendered.
• Work within a typed ontology of pricing entities to ensure model outputs are structured and auditable.
• Identify and remediate systemic latency, data drift, and cold-start issues in the pricing loop.

Skills

LLM depth

Eval harnesses

Observability

MCP tooling

LangChain

LlamaIndex

Typed ontology

GDPR/data residency

Communication

Tools

LangChain

LlamaIndex

Braintrust

OpenRouter

Job description

About the Role

This is a founding engineering role at an early-stage B2B SaaS pricing intelligence company, where you'll own the evaluation systems, feedback loops, and LLM infrastructure that connect model outputs directly to revenue and compliance outcomes. You'll join a small, product-focused team shipping to real customers - with meaningful influence over the technical direction of the AI platform from day one.

What You'll Do

• Build eval harnesses and benchmarks using tracked pricing outcomes as ground truth for model quality.

• Systematize and automate expert review workflows that are currently handled manually.

• Develop AI personas that simulate B2B buying committees using usage data and call transcripts.

• Automate persona training pipelines and reduce reliance on manual processes.

• Own LLM routing across multiple providers with explicit cost, latency, and quality tradeoffs.

• Maintain infrastructure and data residency boundaries (e.g., keeping EU model calls within the EU).

• Extend the MCP server used by LLM agents so that features are agent-driven, not just UI-rendered.

• Work within a typed ontology of pricing entities to ensure model outputs are structured and auditable.

• Identify and remediate systemic latency, data drift, and cold-start issues in the pricing loop.

What We're Looking For

• 8+ years of engineering experience with strong, recent production LLM depth.

• Proven track record shipping and owning LLM-powered product features end-to-end - from development through production monitoring.

• Hands-on experience building eval harnesses and observability for LLM systems, including baselining prompts against a typed ontology for release gating.

• Experience with MCP or building tools for LLM agents; familiarity with platforms such as LangChain, LlamaIndex, Braintrust, or OpenRouter is a plus.

• Experience with structured data models and typed ontologies (e.g., Pydantic) to land model outputs in auditable, structured form.

• Background operating under data residency, SOC2, and GDPR constraints - ideally in pricing, billing, or payments domains where accuracy is audited.

• Strong communication skills to explain non-deterministic systems to clients, pricing experts, and non-technical stakeholders.

• Comfortable with ambiguity and high ownership in a lean, fast-moving startup environment.

• Must be authorized to work in the United States; visa sponsorship is not available.

Compensation & Benefits

Salary range: $225,000 - $255,000 USD annually. Visa sponsorship is not available.

Location

On-site in Amsterdam, Netherlands.