Senior Data Quality Engineer - Data Reliability & Scraping

develop; Greater London; Hybrid; GBP 51,000 - 85,000; Full time

Benefits offered by this job

37 days annual leave

Hybrid working

Home-working equipment

£500 learning & development budget

Team events & monthly lunches

Job summary

develop in London is seeking a Senior Data QA Engineer to own data quality, accuracy, and reliability across its large-scale property data platform. The role focuses on strengthening scraping resilience and building practical tools to resolve issues before customers see them.

You will design data quality checks, maintain scraping infra (Playwright, proxies, anti-bot), and orchestrate data workflows with Dagster or similar.

Qualifications

• Strong Python and SQL skills with dbt and BigQuery experience.
• Proven experience designing data quality, validation and anomaly detection processes.
• Production experience with web scraping at scale, ideally using Playwright or equivalent.
• Experience with big data sets and big data warehouses.
• Experience with proxies, anti-bot measures and scraper monitoring.
• Experience with Dagster, Airflow, Prefect or another data orchestrator.
• Comfort working with TypeScript/Node.js.
• Hands-on, pragmatic problem-solver with strong initiative.
• Knowledge of the UK property/planning sector.

Responsibilities

• Design and develop data quality checks covering validation, anomaly detection, freshness and coverage.
• Own and improve production web-scraping infrastructure, including Playwright, proxies and anti-bot handling.
• Build and maintain data orchestration workflows using Dagster or similar.
• Develop internal QA and investigation tools to identify and resolve data issues.
• Use Python, SQL, dbt and BigQuery to diagnose root causes and improve data reliability.
• Work with TypeScript/Node.js where required.
• Collaborate with data and engineering teams to raise the quality bar.
• Leverage AI-assisted tools to build and ship solutions efficiently.

Skills

Python

SQL

BigQuery

Data quality

Problem solving

Tools

dbt

Dagster

Airflow

Prefect

Playwright

Node.js

Job description

develop in London is seeking a Senior Data QA Engineer to own data quality, accuracy, and reliability across its large-scale property data platform. The role focuses on strengthening scraping resilience and building practical tools to resolve issues before customers see them.

You will design data quality checks, maintain scraping infra (Playwright, proxies, anti-bot), and orchestrate data workflows with Dagster or similar.