Senior Data Infra Engineer: Scalable AI Pipelines

News Corporation; Dublin; On-site; EUR 86,000 - 95,000; Full time

Job summary

Storyful is seeking a Senior Data Infrastructure Engineer to build the foundations for scalable AI-enabled products. You will design ingestion, processing, storage, and serving layers that power future data and AI features.

This hands-on role combines data engineering depth with across-AI infrastructure responsibilities, owning pipelines, observability, and cost-aware processing while collaborating with ML engineers, software, and product managers.

Qualifications

• Strong experience in data engineering or platform engineering in production environments.
• Excellent Python skills and solid SQL fundamentals.
• Experience building reliable ingestion and transformation pipelines at scale.
• Strong understanding of data modeling across structured and unstructured datasets.
• Experience with workflow orchestration tools such as Airflow, Dagster, Prefect, Temporal, or equivalent.
• Strong cloud engineering experience in AWS, GCP, or Azure, with clear transferability across platforms.
• Experience with infrastructure as code and modern deployment practices.
• Experience with distributed systems, event-driven patterns, and data-intensive applications.
• Familiarity with search, vector, or retrieval systems used in AI-backed products.
• Ability to work cross-functionally and act as a technical leader without losing hands-on depth.

Responsibilities

• Work closely with ML engineers, software engineers, product managers, and leadership to turn raw structured and unstructured data into trustworthy product capabilities.
• Design and build scalable batch and streaming pipelines for structured, semi-structured, and unstructured data
• Own the ingestion and processing architecture for documents, text, metadata, and other content sources
• Build robust data workflows for parsing, chunking, enrichment, indexing, and retrieval
• Create the platform foundations for AI products, including orchestration, data quality, observability, lineage, and cost-aware processing
• Design storage patterns across object stores, relational databases, search/vector systems, and where appropriate graph or knowledge-based systems
• Partner with ML and product engineering to productionise AI features, agentic workflows, and retrieval-backed user experiences
• Define data contracts, schema evolution practices, and quality controls across services and teams
• Improve reliability, freshness, and traceability of pipelines that feed customer-facing products
• Contribute hands-on code while helping set engineering standards and mentoring other engineers

Skills

Data engineering

Python

SQL

Data modeling

Workflow orchestration

Cloud infrastructure

IaC

Distributed systems

Search/Vector systems

Technical leadership

Job description

Storyful is seeking a Senior Data Infrastructure Engineer to build the foundations for scalable AI-enabled products. You will design ingestion, processing, storage, and serving layers that power future data and AI features.

This hands-on role combines data engineering depth with across-AI infrastructure responsibilities, owning pipelines, observability, and cost-aware processing while collaborating with ML engineers, software, and product managers.