Senior Full Stack Data Engineer

Hive Science LLC; Greater London; Híbrido; GBP 90.000 - 130.000; Jornada completa

Descripción de la vacante

Hive Science LLC is seeking an experienced Data Engineer to design and build the data infrastructure behind Hive's enrichment products. You will join the product and data engineering teams to deploy scalable pipelines that ingest, transform and link external datasets with Hive’s core platform.

You’ll ensure repeatable onboarding, robust data contracts and production-grade validation across datasets. The role emphasizes building extensible systems, enabling rapid onboarding of new data sources

Formación

• Experience designing and building production data infrastructures.

Responsabilidades

• Build infrastructure to ingest, link, transform, enrich and deliver Hive intelligence across providers and customers.
• Create repeatable onboarding for customer data and integrate with Hive's data architecture.
• Design for rapid onboarding of new datasets, schemas, grains and keys.
• Develop reusable components for record linkage, mapping and schema validation.
• Automate QA for drift, coverage loss and geographic mismatches with lineage tracking.
• Collaborate with data engineering to improve reliability and scale.

Conocimientos

Python programming

SQL proficiency

Bash scripting

Problem solving

communication

Educación

BSc in Computer Science or related

Herramientas

Airflow

Dagster

Prefect

AWS

Snowflake

Databricks

Docker

Terraform

CloudFormation

CI/CD (GitHub Actions)

Descripción del empleo

Hive Science is a tech start-up whose technologies are in use by many of the world’s largest brands including Land Rover, Edward Jones & numerous other fortune 500 brands. The Hive platform is delivering novel intelligence at the intersection of quantitative social psychology, behavioral science & AI/ML to accelerate & transform how work is done. We’re looking for someone to join our team and get on board this rocket ship with us..

About the role:

You'll help design & drive the data infrastructure behind Hive's enrichment products: how Hive intelligence is joined to external client &partner datasets. You'll design &build extensible, repeatable systems and integration capabilities to allow for the same core intelligence to append to partner & customers' data. You'll join the product team and work alongside our existing data engineering team, which runs Hive's core platform. You'll move quickly, build what's needed and harden it for production

• Build infrastructure that ingests, links, transforms, enriches and delivers Hive intelligence across external data providers and customers, with bidirectional flows in each destination's required format.
• Build repeatable onboarding for customer first-party data, covering intake, validation, standardisation and integration with Hive's existing data architecture.
• Design for rapid onboarding of new datasets, schemas, grains and keys, with clear boundaries between core platform and partner-specific configuration.
• Build reusable record linkage, key mapping, schema mapping, data contracts, validation and output generation across audience, segment and geographic grains.
• Automate QA for coverage loss, join-rate changes, schema drift and geographic mismatches, with lineage from source to final output.
• Partner with our data engineering team to strengthen shared infrastructure and pipelines for reliability, monitoring and scale.

Requirements:

• Production data platforms with high-volume ETL/ELT, distributed processing and orchestration (e.g. Airflow, Dagster, Prefect), as distinct from analytics pipelines.
• Python and SQL at a high level, plus Bash automation.
• Cloud infrastructure, with AWS preferred (S3, Lambda, ECS, IAM); Snowflake and/or Databricks; Docker; CI/CD (GitHub Actions or similar); Infrastructure-as-Code (Terraform or CloudFormation).
• Record linkage and entity matching across heterogeneous datasets.
• Data clean-room set-up and integration (Snowflake, AWS Clean Rooms, InfoSum, Databricks or similar).
• Geospatial data engineering: spatial SQL, geographic indexing, Census geographies, crosswalks and large-scale spatial joins.
• Database design and scaling: SQL, NoSQL, vector DBs (Pinecone, FAISS), graph DBs (Neo4j).
• Privacy and security engineering: aggregation thresholds, suppression rules, role-based access, encryption at rest and in transit, secrets management, lineage and audit.
• Automated external data acquisition (APIs, feeds, scraping) built for continuous updates rather than one-off pulls.
• Monitoring and alerting across models and infrastructure (CloudWatch, Prometheus/Grafana, Sentry or similar; MLOps).
• AI and ML integration: productionising models (embeddings, LLMs); orchestrating LLM and NLP pipelines; ETL for training and inference data.
• Fluency with Claude Code: you use AI coding agents daily, and they make you meaningfully faster.
• External data providers, consumer or audience data, or data marketplaces
• A degree in Computer Science, Engineering, Mathematics or a related field, or equivalent experience.

Nice to have

• Synthetic data generation and validation for testing and privacy-preserving development.
• MCP integration with data sources and agents.
• Frontend basics (React/TypeScript) for internal tools and QA dashboards.
• Early-stage or high-growth startup experience.

Additional Information:

About you

You may come from a research lab, or from a startup where you wore every hat. You care about clean, reliable, well-documented systems, and you like making hard data problems boring.

As a fast-paced startup, every day is different. We're nimble and creative, and we value intellectual humility. We work hard because we're dedicated to the future we're building, and our team is great. At Hive we only hire exceptional people, so you'll be surrounded by passionate, smart people building the future of customer intelligence. You'll thrive here if you:

• Are comfortable with competing demands and multiple priorities at once
• Have a \"solve the problem\" mentality
• Are scrappy and creative
• Share a passion for the Hive Science mission and love the scientific method
• Location: Hybrid, 3 days a week in our London or Durham office. You must live within commuting distance.
• Employment type: Full-time.
• Work authorization: You must have an existing right to work in the UK or US, depending on the location you apply for. We're not able to offer visa sponsorship for this role.

Why do you have a passion for the work that you do? What excites you about combining your skills with quantitative social psychology & behavioral science?

Tell us about a time when your work impacted customer success: