Senior Data Platform Engineer - Data Products & Architecture

Francisco Partners; Belfast City District; Hybrid; GBP 90,000 - 130,000; Full time

Job summary

Black Duck Software, Inc. in Belfast, UK seeks a Senior Data Platform Engineer to design and operate a scalable data platform across product lines. You will define canonical customer identifiers, build batch and streaming ingestion pipelines, and own data reliability—contracts, quality, lineage, and monitoring.

Collaboration with product, engineering, and security teams is essential. You will design data models for operational and analytics stores, power ML workflows with trusted datasets, and

Qualifications

• Significant experience building and operating production data platforms at scale.
• Strong SQL and Python skills used to build pipelines and automation.
• Experience with AWS and Google Cloud IaaS.
• Experience with both operational databases (RDS) and analytics stores (OLAP).
• Data modeling expertise with conformed dimensions and a single source of truth.
• Track record delivering data products with clear contracts and reliability.
• Ability to balance latency, accuracy, cost, and security.
• Familiarity with lakehouse patterns and open table formats.
• Experience with batch + real-time orchestration and backfills.
• Familiarity with ML data needs and AI-adjacent workflows.

Responsibilities

• Lead design and build-out of cross-product data services from one governed data plane.
• Define the customer data plane model: canonical customer identifiers, shared dimensions, and consistent facts used across products.
• Build ingestion patterns for batch, streaming, and event data, with repeatable onboarding for new sources.
• Own the operational playbook for data reliability: data contracts, quality checks, lineage, monitoring, and incident response.
• Implement and run access methods that make data usable: curated datasets, secure query interfaces, and product-ready data APIs where needed.
• Productize customer-facing data products (datasets, metrics, exports, and feeds) with versioning, documentation, and clear ownership.
• Design data models that fit both operational systems (RDS) and analytics stores (columnar/OLAP), including performance and cost tuning.
• Ensure data products also power ML workflows: trusted training datasets, feature-ready outputs, and consistent definitions for decision-making.
• Enable AI automation by delivering reliable, low-latency, governed data products that can be used safely in automated workflows.
• Partner closely with product, engineering, and security stakeholders to align data products to roadmap priorities and customer outcomes.
• Raise the technical bar through architecture reviews, standards, and mentoring—while staying hands-on in key systems.

Skills

SQL

Python

Cloud AWS

Google Cloud

Data modeling

Production data platforms

On-call ownership

ML data needs

Lakehouse patterns

Batch + streaming

Backfills

Tools

Airflow

RDS

OLAP databases

Job description

Black Duck Software, Inc. in Belfast, UK seeks a Senior Data Platform Engineer to design and operate a scalable data platform across product lines. You will define canonical customer identifiers, build batch and streaming ingestion pipelines, and own data reliability—contracts, quality, lineage, and monitoring.

Collaboration with product, engineering, and security teams is essential. You will design data models for operational and analytics stores, power ML workflows with trusted datasets, and