Senior Data Scientist (Agent Evaluation & Optimisation)

Swap; Greater London; On-site; GBP 70,000 - 90,000; Full time

Job summary

A leading tech firm in London is seeking a Senior Data Scientist to enhance their AI agents. This hands-on role involves architecting agent systems, implementing memory systems, and building evaluation frameworks. Candidates should have 3+ years of data-driven optimisation experience, strong Python skills, and familiarity with voice systems. The position offers competitive PTO, wellness benefits, and stock options, contributing to a culture of diversity and equality.

Benefits

Stock options in a high-growth startup

Competitive PTO with public holidays

Private Health

Wellness benefits

Breakfast Mondays

Qualifications

• 3+ years experience driving data-driven optimisation.
• Solid experience in building measurement frameworks and test sets.
• Strong Python engineering skills and ability to integrate with APIs.

Responsibilities

• Design and evolve agent systems and components.
• Build evaluation harnesses to measure quality and reliability.
• Implement and iterate on memory systems and workflows.

Skills

Data-driven optimisation

Evaluation mindset

System and architecture thinking

Python engineering

Experience with voice systems

TypeScript willingness to learn

E-commerce familiarity

Job description

London

About Swap

Swap is the infrastructure behind modern agentic commerce. The only AI-native platform connecting backend operations with a forward-thinking storefront experience.

Built for brands that want to sell anything—anywhere, Swap centralizes global operations, powers intelligent workflows, and unlocks margin‑protecting decisions with real‑time data and capability. Our products span cross‑border, tax, returns, demand planning, and our next‑generation agentic storefront, giving merchants full transparency and the ability to act with confidence.

At Swap, we’re building a culture that values clarity, creativity, and shared ownership as we redefine how global commerce works.

About the Role

As a Senior Data Scientist, you’ll help build the intelligence layer behind Swap’s agentic storefront, making our AI agents feel helpful, consistent, and trustworthy. This is a hands‑on role focused on agent components like memory, multimodal or VTO workflows, and our voice agent. You’ll work closely with product and engineering and have real ownership over what we build and Stair how it evolves.

Key responsibilities

• Agent architecture & component design: Design and evolve the agent system and its components (memory, tool use, workflows), with clear interfaces and reliable behaviour. Partner with engineering to integrate retrieval and search capabilities into agent flows.
• Evaluation‑first improvement: Build evaluation harnesses (offline and online) to measure quality, reliability, and task completion. Define rubrics, golden sets, regression tests, and lightweight dashboards. Run experiments (A/B tests where appropriate) and translate results into concrete product improvements.
• Memory systems (Python): Implement and iterate on memory. Capture relevant user and context signals, improve retrieval and selection, and evaluate usefulness and failure modes. Establish feedback loops to learn from real interactions and systematically reduce errors.
• Multimod tientallen or VTO (Python): Iterate on visual and multimodal workflows using APIs. Evaluate outputs, identify failure patterns, and improve prompts, tools, and data. Build pragmatic test sets for multimodal૩ quality and user‑perceived outcomes.
• Voice agent experience (TypeScript and product sensibility): Help develop the voice agent. Evaluate the end‑to‑end experience, craft the persona, add tools, call APIs, and iterate. You don’t need to be a TypeScript expert, but you should be willing to work in it and learn fast. concursos;ol>
• Prompting & tool reliability: Create and refine prompts, tool schemas, and structured outputs to reliably execute multi‑step workflows. Add practical guardrails such as fallbacks, error handling, logging and tracing hooks, and regression checks.

What we would like to see

Must‑haves

• 3+ years experience driving data‑driven optimisation (GenAI products and/or data science problems), with a track record of shipping improvements.
• Strong evaluation mindset. You have built measurement frameworks, test sets, or experimentation loops that make systems measurably better.
• Solid system and architecture thinking for agent‑like products, including clear interfaces, iterative improvement, and pragmatic tradeoffs.
• Strong Python engineering, and comfortable integrating with APIs. You can build quick, reliable prototypes and harden them over time.
• Comfort working closely with product and engineering and owning outcomes end‑to‑end.
• Experience with voice systems, multimodal workflows, or building agent tools.
• Some TypeScript experience, or strong willingness to learn.
• E‑commerce or marketplace familiarity.
• Stock options in a high‑growth startup
• Competitive PTO with public holidays additional
• Private Health
• Wellness benefits
• Breakfast Mondays

Diversity & Equal Opportunities

We embrace diversity and equality in a serious way. We are committed to building a team with a variety of backgrounds, skills, and views. The more inclusive we are, the better our work will be. Creating a culture of equality isn’t just the right thing to do; it’s also the smart thing.