Senior Full Stack Data Engineer

Hive Science LLC; Durham (NC); Híbrido; USD 120 000 - 180 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Hybrid work model

Office in Durham

Resumo da oferta

Hive Science LLC is seeking a data infrastructure engineer to design and build scalable systems that connect Hive intelligence with partner datasets. You will contribute to the data platform, integration flows, and QA automation.

You will collaborate with our data engineering team to ensure reliable, production-ready pipelines, API endpoints, and data contracts for multi-source enrichment. This hybrid role is based in Durham, NC with UK options.

Qualificações

• Strong experience building data infrastructure for ingestion, linking, transformation, enrichment and delivery.
• Experience onboarding customer data, validating, standardising and integrating with existing architecture.
• Proven ability to design scalable data contracts, mappings and output generation across grains and keys.
• Hands-on with QA, lineage tracking, and monitoring for reliability and scale.

Responsabilidades

• Build infrastructure to ingest, link, transform, enrich and deliver Hive intelligence across providers and customers.
• Create repeatable onboarding for first-party data with intake, validation and integration.
• Design interfaces between core platform and partner-specific config with clear boundaries.
• Develop reusable record linkage, mapping, schema mapping and data contracts.
• Automate QA for schema drift, join-rate changes, and geographic mismatches with lineage.
• Collaborate with data engineering to strengthen pipelines for reliability and scale.
• Maintain production data platforms with high-volume ETL/ELT and orchestration (Airflow, Dagster, Prefect).
• Develop APIs (FastAPI/Flask) and CI/CD workflows; work with AWS services and Snowflake/Databricks.
• Implement data clean-room setups (Snowflake, AWS Clean Rooms, Databricks).
• Address privacy and security: access controls, encryption, secrets management, audits.

Conhecimentos

Python

SQL

Bash

AWS

Snowflake

Databricks

Docker

Airflow

Dagster

API development

Geospatial processing

NoSQL

Vector databases

Neo4j

MLOps

Claude Code

Data governance

GitHub Actions

Formação académica

Bachelor's degree in CS/Engineering or related field

Ferramentas

FastAPI

Flask

GitHub Actions

Terraform

Descrição da oferta de emprego

You'll help design & drive the data infrastructure behind Hive's enrichment products: how Hive intelligence is joined to external client & partner datasets. You'll design & build extensible, repeatable systems and integration capabilities to allow for the same core intelligence to append to partner & customers' data. You'll join the product team and work alongside our existing data engineering team, which runs Hive's core platform. You'll move quickly, build what's needed and harden it for production

• Build infrastructure that ingests, links, transforms, enriches and delivers Hive intelligence across external data providers and customers, with bidirectional flows in each destination's required format.
• Build repeatable onboarding for customer first-party data, covering intake, validation, standardisation and integration with Hive's existing data architecture.
• Design for rapid onboarding of new datasets, schemas, grains and keys, with clear boundaries between core platform and partner-specific configuration.
• Build reusable record linkage, key mapping, schema mapping, data contracts, validation and output generation across audience, segment and geographic grains.
• Automate QA for coverage loss, join-rate changes, schema drift and geographic mismatches, with lineage from source to final output.
• Partner with our data engineering team to strengthen shared infrastructure and pipelines for reliability, monitoring and scale.

• Production data platforms with high-volume ETL/ELT, distributed processing and orchestration (e.g. Airflow, Dagster, Prefect), as distinct from analytics pipelines.
• Python and SQL at a high level, plus Bash automation.
• Cloud infrastructure, with AWS preferred (S3, Lambda, ECS, IAM); Snowflake and/or Databricks; Docker; CI/CD (GitHub Actions or similar); Infrastructure-as-Code (Terraform or CloudFormation).
• Record linkage and entity matching across heterogeneous datasets.
• Data clean-room set-up and integration (Snowflake, AWS Clean Rooms, InfoSum, Databricks or similar).
• Geospatial data engineering: spatial SQL, geographic indexing, Census geographies, crosswalks and large-scale spatial joins.
• Database design and scaling: SQL, NoSQL, vector DBs (Pinecone, FAISS), graph DBs (Neo4j).
• Privacy and security engineering: aggregation thresholds, suppression rules, role-based access, encryption at rest and in transit, secrets management, lineage and audit.
• Automated external data acquisition (APIs, feeds, scraping) built for continuous updates rather than one-off pulls.
• API development (FastAPI/Flask, REST/GraphQL).
• Monitoring and alerting across models and infrastructure (CloudWatch, Prometheus/Grafana, Sentry or similar; MLOps).
• AI and ML integration: productionising models (embeddings, LLMs); orchestrating LLM and NLP pipelines; ETL for training and inference data.
• Fluency with Claude Code: you use AI coding agents daily, and they make you meaningfully faster.
• External data providers, consumer or audience data, or data marketplaces.
• A degree in Computer Science, Engineering, Mathematics or a related field, or equivalent experience.

Nice to have

• Synthetic data generation and validation for testing and privacy-preserving development.
• MCP integration with data sources and agents.
• Frontend basics (React/TypeScript) for internal tools and QA dashboards.
• Early-stage or high-growth startup experience.

About You

You may come from a research lab, or from a startup where you wore every hat. You care about clean, reliable, well-documented systems, and you like making hard data problems boring.

As a Fast-paced Startup, Every Day Is Different. We're Nimble And Creative, And We Value Intellectual Humility. We Work Hard Because We're Dedicated To The Future We're Building, And Our Team Is Great. At Hive We Only Hire Exceptional People, So You'll Be Surrounded By Passionate, Smart People Building The Future Of Customer Intelligence. You'll Thrive Here If You

• Are comfortable with competing demands and multiple priorities at once
• Have a "solve the problem" mentality
• Are scrappy and creative
• Share a passion for the Hive Science mission and love the scientific method

Important details

• Location: Hybrid, 3 days a week in our London or Durham office. You must live within commuting distance.
• Employment type: Full-time.
• Work authorization: You must have an existing right to work in the UK or US, depending on the location you apply for. We're not able to offer visa sponsorship for this role.