Senior Full Stack Developer — Evaluation & Model Quality

Delta Labs; Zürich; Vor Ort; CHF 120.000 - 170.000; Vollzeit

Benefits dieser Stelle

Ownership of systems

Direct work with a research team

Early-stage company impact

Zusammenfassung

Delta Labs in Zürich, Switzerland is seeking a software engineer to build the measurement layer for behavioural simulation and to ensure the system can scale testing and evaluation inside the release loop.

You will own the observability of models, create pipelines to compare simulated behaviour with real data, and translate research judgments into runnable code, partnering with a small, focused team.

Qualifikationen

• 4+ years building production software in a professional setting.
• Backend services, data pipelines, and automation workflows.
• Statistical literacy to discuss with researchers and justify decisions.
• LLM fluency to reason about model improvements vs. metrics.
• Ability to own end-to-end features and deliverables.

Aufgaben

• Build the measurement layer: evaluation infrastructure, versioned datasets, provenance, holdout integrity, regression suites that run inside the development loop.
• Build pipelines comparing simulated responses against real behavioural data (transaction data, panel data, client validation studies).
• Turn research judgment into running code: define validity criteria, create metrics, rubrics, automated graders, release criteria.
• Own model observability: extend LLM tracing to trace regressions to prompts, models or populations.
• Develop tools for behavioural scientists to inspect outputs, label data, and investigate regressions without involving engineers.

Kenntnisse

Backend development

Data pipelines

Testing & maintenance

LLM fluency

AI tooling

Ownership

Tools

TypeScript

PostgreSQL

Python

Jobbeschreibung

The most important question anyone can ask about a simulated population is whether it's right. It's the question our clients ask, and the one we hold ourselves to. Answering it rigorously, repeatably and at scale is what this role exists to do.

You'd be building the measurement layer for behavioural simulation: the systems that decide whether a model change improved the thing we care about, whether a population behaves like the people it represents, and whether an output is good enough to put in front of a client making a real decision. You'd work alongside a research team with unusual depth in exactly this problem — survey methodology, experimental design, psychometrics, computational models of decision-making — turning measurement judgment that today lives in their heads into infrastructure that runs on every release.

An increasing share of that work is validating simulated behaviour against real behavioural outcomes, not just self-reported response. That's the frontier of this role.

RESPONSIBILITIES

• – Build the measurement layer: evaluation execution infrastructure, versioned datasets, provenance, holdout integrity, regression suites that run inside the development loop rather than as a gate at the end
• – Build the pipelines comparing simulated responses against real behavioural data — transaction records, panel data, client validation studies
• – Turn research judgment into running code: take a validity criterion and make it a metric, a rubric, an automated grader, a release criterion
• – Own model observability — extend our LLM tracing so a regression can be traced to the prompt version, model or population that caused it
• – Build tools that let a behavioural scientist inspect outputs, label data and investigate a regression without asking an engineer

YOU MAY BE A FIT IF

• – 4+ years building production software, with sound judgment on system design, testing and maintainability — this is an engineering role
• – Backend services, data pipelines, relational modelling, automation workflows
• – Statistical literacy: sampling, distributions, uncertainty, calibration, confidence intervals. Enough to build the right thing and argue with a researcher productively — you needn't be a statistician, you need to not be intimidated by one
• – Enough LLM fluency to reason about whether a change improved the thing we care about or just moved a number
• – AI tools as standard development practice
• – Hands-on ownership: build the first version, inspect the data, find the metric was wrong, revise it — repeatedly

STRONG CANDIDATES MAY ALSO HAVE

• – LLM or ML evaluation systems, benchmark platforms, regression suites, experiment tracking, model-quality dashboards
• – LLM-as-judge systems, rubric design, grader calibration, human-in-the-loop labelling
• – Survey methodology, psychometrics, causal inference or A/B testing infrastructure
• – Internal tooling for researchers or data scientists
• – Sensitive human data: consent, access control, auditability

ABOUT DELTA LABS

Delta Labs uses AI to simulate and predict consumer behaviour at scale. We build Elaiia, a simulation engine that generates AI Twins — intelligent synthetic agents that mirror real consumer populations. Our clients use Elaiia to simulate customer decisions before committing to them: pricing strategies, product launches, campaign messaging, channel allocation. We replace surveys, focus groups, and intuition with simulation-based evidence.

We're a small, focused team and we intend to stay that way. We give people ownership, trust, and the autonomy to do their best work. We work with urgency and intellectual honesty and expect new team members to match our pace. We seek individuals who are curious, rigorous, and want their work to have demonstrable impact. If you're drawn to the idea of a small team building something that hasn't existed before, let's build together.

THE STACK

TypeScript frontend (Next.js/React), PostgreSQL, Python microservices, durable background jobs, integrated LLM APIs (OpenAI, Anthropic, Google Gemini), LangGraph agents with tracing. Deployed on Vercel and Microsoft Azure.

LOCATION

This role is based in Zürich, Switzerland. Delta Labs is an in-person company. Candidates are expected to be located in the Zürich area or open to relocation.

BENEFITS

Ownership of systems rather than tickets. Direct work with a research team of unusual depth — behavioural scientists, experimental economists, psychometricians and cognitive scientists whose methods you'd be building. A product global enterprises use for decisions that matter. The chance to shape an early-stage company.

Delta Labs is an equal opportunity employer, welcoming applicants of all backgrounds.