Founding Engineer, Applied AI Research

Armature (YC P26); Paris; On-site; EUR 70,000 - 120,000; Full time

Job summary

Armature (YC P26) invites an engineer to own the research we publish and the evals system behind our results. You will collaborate with our forward deployed engineer and our product engineer to drive studies and build the evaluation framework that underpins our products.

You will design and run studies on how coding agents select tools, compare model behaviors, and translate findings into actionable improvements for our dashboards and product features.

Qualifications

• Engineer from a top school.
• Internships show you can build things.
• Ability to go from raw data to a finding.
• You come with your own ideas, and you like to test them.
• High agency: you move forward without waiting for instructions.

Responsibilities

• Publish research on how coding agents think and choose tools and publish findings.
• Design and run studies on tool usage, model differences, and data harnesses.
• Automate research runs and analysis to improve speed.
• Write up findings so developers want to read and share them.
• Work on the evals system to test and ship improvements to product.

Skills

Coding agents

Research

Data analysis

Experiment design

Education

Top engineering school

Job description

Location: Paris, France. We value working in person, and we stay flexible. You may also get the chance to join us on occasional trips to San Francisco.

Language: English is required. You do not need to speak French.

The role

You own two things at Armature: the research we publish and the evals system behind our results. You work on both hand in hand with our forward deployed engineer and our product engineer.

• The research we publish. Our studies on how coding agents think and choose are how most customers find us. You decide what we study next, you run the studies behind our public leaderboards, and you publish the findings under your name.
• The evals system. Everything we tell a customer depends on our evals, so they have to be excellent. You research how to design them to get the best results, and you build what works into the product.

This field is young, so many of your questions won't have an answer yet, and what you find will shape how software gets rebuilt for agents.

What you will do

Published research

• Design and run studies on how coding agents choose tools, how models and harnesses differ, and how agents use software.
• Find the next questions worth asking, and new ways to look at the data we already have.
• Automate the research runs and the analysis, so each study takes less time than the one before.
• Write up your findings so developers want to read and share them.

Evals system

• Test the latest research on evals against our own data, and keep what improves our results.
• Turn the questions our customers ask into better evals.
• Ship what works into the product with our product engineer.

Who we look for

• You're an engineer from a top school, and if you're a new grad, your internships show that you can build things.
• You build through coding agents every day: you split the work, guide the agents and understand what they produce, and this is the skill we value most.
• You can go from raw data to a finding on your own.
• You come with your own ideas, and you like to test them.
• You have high agency: you move forward without waiting for instructions.

Not a fit if

• You want pure research and nothing to ship.
• You need a fixed scope before you start.
• You want to train models more than you want to study how they behave.
• You prefer to write every line yourself over working through coding agents.