Lead Data Engineer

flagshippioneeringinc; Cambridge; On-site; GBP 90,000 - 120,000; Full time

Job summary

Flagship Pioneering invites applications for a Lead Data Engineer within its Scientific Cloud group in Cambridge. The role focuses on modernizing data infrastructure, building scalable pipelines, and establishing best practices across data platforms.

You will collaborate with cross-functional teams spanning I&O, Lab IT, and data science, driving innovation through robust, observable systems. You will mentor engineers, set coding and deployment standards, and apply AI-assisted approaches to

Qualifications

• 6+ years in data engineering, preferably life sciences or healthcare.
• Design and operate cloud-native AWS data architectures.
• Proficient in Python and SQL for pipelines and automation.
• Experience with Dagster, dbt, Spark, Iceberg, Lake Formation, Athena, Glue.
• Strong Git, CI/CD, CDK, Docker, ECS experience.
• Experience handling structured and unstructured data (CSV/JSON/Parquet).

Responsibilities

• Act as technical lead on complex data engineering projects across interdependent systems.
• Lead the implementation of data infrastructure, platforms, tools, and data products.
• Improve reliability of data storage solutions (lakehouses, marts, warehouses).
• Build data pipelines for data lakes, marts, and platforms.
• Set engineering standards for testing, docs, release management, and monitoring.
• Oversee platform observability, resilience, capacity, and performance.
• Diagnose failures and lead incident reviews to drive fixes.
• Collaborate with I&O, Lab IT, PI Tech, data science and bioinformatics.
• Mentor junior engineers and establish standards.
• Leverage generative AI tools to accelerate pipeline development.

Skills

Data engineering

Cloud architecture

Python

SQL

Leadership

Tools

Dagster

dbt

Spark

Iceberg

Lake Formation

Athena

Glue

Git

CDK

Docker

ECS

Job description

Who We Are

Flagship Pioneering is a biotechnology company that invents and builds platform companies that change the world. We bring together the greatest scientific minds with entrepreneurial company builders and assemble the capital to allow them to take courageous leaps. Those big leaps in human health and sustainability exponentially accelerate scientific progress in areas ranging from cancer detection and treatment to nature-positive agriculture.

What sets Flagship apart is our ability to advance biotechnology by uniting life science innovation, company creation, and capital investment under one roof in a way that is largely without precedent. Our scientific founders, entrepreneurial leaders, and professional capital managers are each aligned around an institutionalized process that enables us to innovate and transform for the benefit of people and planet.

Many of the companies Flagship has founded have addressed humanity's most urgent challenges: vaccinating billions of people against COVID-19, curing intractable diseases, improving human health, preempting illness, and feeding the world by improving the resiliency and sustainability of agriculture.

Flagship has been recognized twice on FORTUNE's \"Change the World\" list, an annual ranking of companies that have made a positive social and environmental impact through activities that are part of their core business strategies, and has been twice named to Fast Company's annual list of the World's Most Innovative Companies.

About Scientific Cloud

Scientific Cloud is Flagship Pioneering's portfolio-facing IT organization, responsible for the cloud engineering, data and informatics engineering, research systems, lab systems, and vendor management capabilities that power Flagship's emerging companies. The team operates with a portfolio-first orientation - building durable, shared infrastructure that individual ventures can rely on at every stage of company formation and growth.

We are seeking a Lead Data Engineer to lead complex initiatives aimed at modernizing and professionalizing our data infrastructure, platforms, and pipelines. You will lead the implementation of core platform systems and help to set the directions and standards of our data infrastructure and pipelines. You will work closely with other engineers and stakeholders across Infrastructure & Operations (I&O), Lab IT, Pioneering Intelligence (PI) Tech, and the internal scientific community to turn evolving scientific needs into secure, reliable and scalable data engineering solutions.

Reporting into the Associate Director, Data Architecture & Engineering, this is a technical, senior individual-contributor role that will champion engineering best practices, build complex data pipelines, lead data platform implementations, and handle the troubleshooting and optimization of data warehouse and storage solutions.

CORE RESPONSIBILITIES

Data & Platform Engineering

• Serve as the technical lead and subject matter expert on complex data engineering projects involving interdependent systems, legacy environments, and emerging cloud-first platforms.
• Lead the implementation of data infrastructure, platforms, tools, and other data products.
• Lead the optimization and reliability of data storage solutions (lakehouses, marts, warehouses, etc.).
• Implement complex backend and data integration pipelines to support data lakes, marts, and other platforms.
• Establish and reinforce engineering standards for testing, documentation, release management, and monitoring across the team.
• Lead implementations for platform observability, management, resilience, capacity, and performance.
• Diagnose and resolve complex failures spanning infrastructure and data pipelines; lead incident reviews and ensure corrective actions improve the wider system.

Cross-Functional Collaboration & Technical Leadership

• Collaborate across I&O, Lab IT, PI Tech, data science, and bioinformatics teams to ensure architectural alignment, reuse of components, and shared understanding of system states and dependencies.
• Mentor junior engineers and establish engineering standards that drive excellence, reproducibility, and innovation across the team.
• Leverage generative AI tools and frameworks to accelerate pipeline development, data transformation, and metadata enrichment at scale.

REQUIRED QUALIFICATIONS

• 6+ years of hands-on experience in data engineering, preferably in life sciences, biotech, or healthcare environments.
• Proven expertise in designing and operating cloud-native data architectures, particularly within AWS.
• Advanced proficiency in Python and SQL for data manipulation, pipeline development, and system automation.
• Extensive experience with modern frameworks and tools such as Dagster, dbt, Spark, Iceberg, Lake Formation, Athena, Glue, etc.
• Strong experience with Git, CI/CD, CDK, containerized workloads, Docker, and ECS.
• Experience supporting both structured and unstructured data (CSV, JSON, Parquet, imaging, etc.).
• De