Lead Data Platform Engineer

Blackduck; Belfast City District; Hybrid; GBP 100,000 - 140,000; Full time

Job summary

Black Duck Software, Inc. is seeking a senior data engineering leader to design and operate a scalable data platform across multiple product lines in a hybrid Belfast setting. You will define a canonical customer data plane, lead ingestion patterns for batch and streaming data, and own data reliability, access, and governance.

You will also drive data products powering ML workflows and AI automation. You will collaborate with product, engineering, and security to align with roadmaps and customer

Qualifications

• Significant experience building and operating production data platforms at scale.
• Strong SQL and Python skills for pipelines, services, and automation.
• Hands-on experience with AWS and Google Cloud (IaaS: compute, storage, networking, IAM).
• Experience with both operational databases (RDS‑style) and analytics stores (columnar/OLAP).
• Strong data modeling, including schema evolution and conformed dimensions.
• Track record delivering data products with clear contracts and reliability.
• Experience with lakehouse patterns and governance.
• Experience with orchestration and streaming systems in production.
• Familiarity with ML data needs and AI-adjacent workflows.

Responsibilities

• Lead design and build-out of cross-product data services from a governed data plane.
• Define the customer data plane model with canonical identifiers and shared dimensions.
• Build and operationalize ingestion patterns for batch, streaming, and event data.
• Own the data reliability playbook: contracts, quality checks, lineage, monitoring, incident response.
• Implement and run secure data access methods and product-ready datasets and APIs.
• Productize data products with versioning, documentation, and ownership.
• Design data models for operational and analytics stores, optimizing performance and cost.
• Support ML workflows with trusted training datasets and feature-ready outputs.
• Collaborate with product, engineering, and security to align with roadmap.

Skills

SQL

Python

AWS

GCP

RDS

OLAP

Data Modeling

Data Products

Data Platforms

Streaming

Orchestration

ML Data

Job description

Black Duck Software, Inc. is seeking a senior data engineering leader to design and operate a scalable data platform across multiple product lines in a hybrid Belfast setting. You will define a canonical customer data plane, lead ingestion patterns for batch and streaming data, and own data reliability, access, and governance.

You will also drive data products powering ML workflows and AI automation. You will collaborate with product, engineering, and security to align with roadmaps and customer