Lead Data Engineer

Obsero; United Kingdom; Remote; GBP 80,000 - 100,000; Full time

Benefits offered by this job

Equity

Job summary

Obsero is seeking a senior data engineer to own the data platform end-to-end, from ingestion to modelling and analytics. You’ll build reliable pipelines, define a single schema across vendors and guide data quality at scale.

You’ll work with the founders and a lead data scientist to implement LLM-driven extraction, sentiment scoring and rapid anomaly detection on a Parquet data lake hosted on AWS. The role combines hands-on engineering with leadership, offers meaningful equity and a salary

Qualifications

• Seven years+ experience owning a data platform end-to-end.
• Strong Python and SQL, comfortable with MongoDB and columnar analytics.
• Experience putting LLM pipelines into production and evaluating them.
• Hands-on ML experience to classify in the pipeline and measure accuracy.
• Experience leading or mentoring engineers.
• Passionate about data quality and challenging numbers when needed.

Responsibilities

• Own data quality checks, monitoring and alerts to catch bad data.
• Maintain coverage as platforms change and new ones launch.
• Provide early warning of data issues and advise on product impact.
• Oversee ingestion from AI platforms and third-party vendors.
• Model data into a single standard schema across vendors.
• Develop LLM-based extraction, classification and sentiment.
• Manage the data layer on AWS and Parquet lake.
• Evaluate and select data tools and partners.
• Lead the data side of engineering, mentoring one engineer.

Skills

7+ years data experience

Python

SQL

MongoDB

LLM pipelines production

ML in production

Leadership/mentoring

Data quality

Tools

Docker

AWS (infrastructure as code)

Job description

The problem

Data is the heart of everything Obsero does, and it has to just work. The AI platforms we track change all the time, often without telling anyone, and when they do we need to know about it first, adapt quickly and keep the numbers correct. We also want to get there efficiently, so we're never paying more for data than we need to. And as new AI platforms launch, we need to be tracking them too, wherever a brand's customers are asking.

That's why we want someone who owns all of this without being asked: building the alerts and systems that catch changes, making the calls on cost and coverage, and coming to us the moment the data tells us something has changed. You'll share the load with the founders and give everything we learn from the data a foundation we can trust.

Why join now

• We've just raised $1.2m, led by SFC Capital, to build specialist AI agents trained on the GEO playbook from a year of client work

• In our first year we captured and analysed over 10 million AI answers, and that number is growing fast

• We're a small team, so you'll shape how the data side is built from the start rather than inherit decisions someone else made

• Meaningful equity, alongside a salary of £80,000 to £100,000

• AI search is one of the fastest-moving areas in marketing, and the data you own is what every client decision rests on

About Obsero

Obsero tracks how brands are cited and recommended across ChatGPT, Gemini, Perplexity, Google AI Mode, Google AI Overviews and Claude, capturing AI answers every day and turning them into the visibility, citation and sentiment data that marketing teams act on.

The role

Everything we sell is built on data, and you will own it. That covers how we capture AI answers from every platform and vendor, how we model them into one consistent schema, how we use LLMs to pull citations, brand mentions and sentiment out of them, and how we know every single day that the numbers clients see are right.

This is a lead role reporting to a founder, so we want someone who looks at the data, tells us plainly when something is not right and comes back with what we need to do about it, rather than waiting for a ticket to arrive. You'll work closely with our lead data scientist, who turns the data you own into the GEO playbook and the recommendations our customers act on.

What you’ll own

• Data quality. The checks, monitoring and alerting that catch broken, stale or inconsistent data before a client does, along with the processes the rest of the team follows to keep it that way

• Coverage. Keeping our tracking current as platforms change and new ones launch, so we can follow a brand wherever its customers are asking

• Early warning. Spotting when the data looks wrong or the platforms behave differently, and telling us what it means for the product and our client work before anyone else notices

• Capture. Ingestion from AI platforms and third-party data vendors, including testing new vendors and switching between them without leaving gaps in a client's history

• Modelling. One standard schema across every vendor and platform, so citation parsing is consistent and a fix is made once rather than vendor by vendor

• AI in the pipeline. LLM-based extraction, classification and sentiment, the machine learning behind how we score and spot anomalies in what we capture, and the evaluations that prove the output is accurate and keep it accurate as the models change underneath us

• The data layer on AWS. Our Parquet data lake, the pipelines that feed it and the query layer that lets clients analyse long date ranges quickly

• Technology partners. Finding, assessing and recommending the vendors, models and tools we build on, from data providers to LLM platforms, and making the case for when to switch

• The data behind our agents. The data, retrieval and evaluation layer our specialist AI agents run on, so every recommendation they make is grounded in data we trust

• Standards. How our engineers work with data, so quality is built into the way we work rather than checked at the end

• People. Leading the data side of engineering, starting with one engineer reporting to you

The problems you’ll take on first

• Real change or model update. Telling a genuine shift in a brand's visibility apart from a model update, quickly enough that clients hear it from us first, working with our lead data scientist on the tests behind it

• Cost. Getting more out of every LLM call and every vendor as the volume of answers we capture keeps growing

• Speed. Keeping the platform fast for clients as the data and the date ranges they analyse get bigger

• Gaps and errors. Detecting gaps, errors and inconsistencies automatically, and normalising the data before it reaches a client

• LLM or code. Deciding when an LLM call is the right way to create data and when code or rules will do the job better, cheaper and more reliably

Our stack

• Python, with Polars, Pandera and Celery

• SQL and MongoDB

• Columnar analytics on a Parquet data lake

• Docker

• AWS, managed as infrastructure as code

What you’ll need

• An experienced data engineer, typically with seven years or more, who has owned a data platform end to end rather than built parts of one

• Strong in Python and SQL, comfortable with MongoDB and columnar analytics, and experienced with queues and idempotent pipelines that can rerun safely without duplicating or losing data

• Someone who has put LLM pipelines into production and knows how to evaluate them, not just prompt them

• Enough hand-on machine learning experience to build classification into the pipeline and measure how accurate it is, so you can judge whether what we capture is right as well as move it around

• Experienced in leading or mentoring other engineers, even if you still want to spend most of your time building

• Passionate and opinionated about data quality, with the confidence to challenge founders and engineers alike when the numbers don't add up

Nice to have

• Agentic pipelines in production

• Experience running AWS infrastructure as code

• Some exposure to TypeScript or modern JavaScript

• Used to startup or scale-up pace, where you decide what matters most and ship it

You’re based in the UK with the right to work here.

How we hire

1. Intro call with the Chief Product Officer, 30 minutes

2. Technical conversation with the Lead Development Engineer

3. Presentation

4. Meet the CEO