Senior Data Platform Engineer – AI & Real-Time Analytics

Anaplan; Greater London; On-site; GBP 90,000 - 130,000; Full time

Job summary

Anaplan is seeking a Principal Data Engineer to lead the data platform across ingestion, transformation, storage, and governance at scale. You will build high-throughput pipelines for real-time and batch processing, enabling enterprise planning workflows and advanced analytics at the foundation of our AI initiatives.

You will shape data systems, optimize queries, and collaborate with analytics, product, and platform teams to maintain data quality and scalability.

Qualifications

• Extensive data engineering experience in complex environments.
• Deep understanding of vector databases, NoSQL, and document stores.
• Hands-on building and shipping large-scale data platforms.
• Experience with Spark, Flink, Hadoop on production systems.
• Strong skills in Kafka, Kinesis and event streaming.
• Proficient in Airflow or Dagster for workflows.
• Cloud data warehouses: Snowflake, BigQuery, Redshift.
• Advanced SQL and Python programming.
• Familiarity with CI/CD, testing and IaC practices.

Responsibilities

• Lead design and deployment of scalable data architectures.
• Build robust ETL/ELT pipelines and data ingestion frameworks.
• Develop data models and governance for analytics metrics.
• Create high-performance data lakes and warehouses for AI workloads.
• Ensure data quality, reliability and observability across assets.
• Collaborate with analytics, product, and platform teams on data use cases.
• Stay current with modern data stack and evaluate new tools.

Skills

Data engineering

Big Data

SQL

Python

Distributed computing

Data architecture

ETL/ELT

CI/CD

Data quality

Tools

Apache Spark

Flink

Hadoop

Kafka

Kinesis

Airflow

Dagster

Snowflake

BigQuery

Redshift

Delta Lake

Job description

Anaplan is seeking a Principal Data Engineer to lead the data platform across ingestion, transformation, storage, and governance at scale. You will build high-throughput pipelines for real-time and batch processing, enabling enterprise planning workflows and advanced analytics at the foundation of our AI initiatives.

You will shape data systems, optimize queries, and collaborate with analytics, product, and platform teams to maintain data quality and scalability.