Senior Data Platform Engineer - Data Products

Black Duck; Belfast City District; Hybrid; GBP 90,000 - 130,000; Full time

Job summary

Black Duck in Belfast, UK, seeks a senior data platform engineer to design and build cross-product data services from a single governed data plane. You will own ingestion, data quality, and data APIs, enabling ML workflows and secure data accessibility.

Bring extensive experience with SQL, Python, AWS/GCP, and data modeling. You will collaborate with product, engineering, and security teams in a hybrid work environment and contribute to data contracts and reliability across the org.

Qualifications

• Significant experience building and operating production data platforms at scale, including on-call and operational ownership.
• Strong SQL and Python skills used to build pipelines, services, and automation.
• Hands-on experience running cloud systems on AWS and Google Cloud (IaaS level: compute, storage, networking, IAM).
• Experience with both operational databases (RDS-style) and analytics stores (columnar/OLAP), including performance tuning.
• Strong data modeling ability, including schema evolution and conformed dimensions.
• Track record delivering data products that other teams depend on, with clear contracts and reliability expectations.
• Ability to balance latency, accuracy, cost, and security without brittle complexity.
• Experience with lakehouse patterns and open table formats, including governance and table maintenance.
• Experience with orchestration and streaming systems used in production, and backfill management.
• Familiarity with ML data needs (training/serving splits, feature-ready datasets).

Responsibilities

• Lead the design and build-out of cross-product data services for multiple product lines from one governed data plane.
• Define the “customer data plane” model: canonical customer identifiers, shared dimensions, and consistent facts used across products.
• Build and operationalize ingestion patterns for batch, streaming, and event data, with repeatable onboarding for new sources.
• Own the operational playbook for data reliability: data contracts, quality checks, lineage, monitoring, and incident response.
• Implement and run access methods that make data usable: curated datasets, secure query interfaces, and product-ready data APIs where needed.
• Productize customer-facing data products (datasets, metrics, exports, and feeds) with versioning, documentation, and clear ownership.
• Design data models that fit both operational systems (RDS) and analytics stores (columnar/OLAP), including performance and cost tuning.
• Ensure data products also power ML workflows: trusted training datasets, feature-ready outputs, and consistent definitions for decision-making.
• Enable AI automation by delivering reliable, low-latency, governed data products that can be used safely in automated workflows.
• Partner closely with product, engineering, and security stakeholders to align data products to roadmap priorities and customer outcomes.
• Raise the technical bar through architecture reviews, standards, and mentoring—while staying hands-on in key systems.

Skills

SQL

Python

AWS

Google Cloud

RDS

OLAP

Data modeling

Data contracts

Data pipelines

Stakeholder collaboration

Job description

Black Duck in Belfast, UK, seeks a senior data platform engineer to design and build cross-product data services from a single governed data plane. You will own ingestion, data quality, and data APIs, enabling ML workflows and secure data accessibility.

Bring extensive experience with SQL, Python, AWS/GCP, and data modeling. You will collaborate with product, engineering, and security teams in a hybrid work environment and contribute to data contracts and reliability across the org.