Principal Data Engineer

Flagship Pioneering, Inc.; Cambridge (MA); On-site; USD 179,000 - 236,000; Full time

Benefits offered by this job

Healthcare coverage

Annual incentive program

Retirement benefits

Broad range of other benefits

Job summary

Flagship Pioneering, Inc. in Cambridge, MA (onsite) seeks a senior data platform leader to evolve a production AWS data platform supporting human genetics, multi-omics, and high-content imaging workflows.

You will own the AWS architecture, implement cost governance, drive DevOps for data pipelines, modernize the platform, and collaborate with ML and computational biology leaders to align data infrastructure with modelling needs. Onsite role with competitive compensation.

Qualifications

• 10+ years building and operating production data infrastructure.
• Deep AWS expertise across S3, IAM, EC2, Batch, Lambda, and Spark.
• Experience as technical owner of production cloud environments, including multi-account architecture.
• Strong Python and SQL for production-grade pipelines.
• Experience with workflow orchestration (Nextflow, Airflow, Temporal, Step Functions, Redun).
• Infrastructure as code using CloudFormation or CDK, plus CI/CD, containers, and observability.
• Data modelling and lakehouse design with metadata management, lineage, and data quality.
• Cost governance at scale across hundreds of terabytes.
• Familiarity with genetics and genomics data.

Responsibilities

• Provide technical ownership and architecture of the AWS data platform.
• Govern platform costs including budgets, tagging, and rightsizing.
• Drive DevOps for data pipelines: IaC, CI/CD, containers, secrets management.
• Modernize and scale data lake, ingestion, and orchestration components.
• Collaborate with ML and computational biology teams to align platform interfaces.

Skills

AWS architecture

Python

SQL

Multi-account AWS

CI/CD

IaC

Data modelling

Cost governance

Workflow orchestration

Cloud security

Tools

S3

IAM

EC2

Batch

Lambda

Spark

EMR

CloudFormation

CDK

CI/CD

Databricks

OpenMetadata

Snowflake

SageMaker

MLflow

Weights & Biases

Job description

Lead the evolution of a production AWS data platform supporting human genetics, multi-omics, and high-content imaging workflows.

• Technical ownership and architecture of the AWS environment, including account structure, IAM, networking, compute, and storage.
• Cost governance for the platform: budgets, alerting, tagging enforcement, rightsizing, and storage lifecycle and tiering.
• Developer operations for the data platform: infrastructure as code, CI/CD, container builds, and secrets and environment management.
• Platform modernization and scaling: upgrade the data lake, managed orchestration, metadata, and data ingestion workflows to support future data production and strengthen lab-to-cloud data transfer.
• Cross-functional partnership with ML and computational biology leaders to define the interface between the platform and modelling.

Requirements

• 10+ years building and operating production data infrastructure, with experience architecting and owning platforms end to end.
• Deep AWS expertise across S3, IAM, EC2, Batch, Lambda, and Spark (EMR or equivalent).
• Experience as technical owner of production cloud environments, including multi-account architecture and cross-account access.
• Strong Python and SQL with a background in production-grade pipeline development.
• Production experience with workflow orchestration such as Nextflow, Airflow, Temporal, Step Functions, Redun, or similar, plus judgement to evaluate and select tools.
• Infrastructure as code using CloudFormation or CDK, plus CI/CD, containers, and observability; DevOps is a core competency.
• Data modelling and warehouse or lakehouse design, including metadata management, lineage, and data quality practices.
• Cost governance at scale including budgets, tagging, rightsizing, and storage lifecycle across hundreds of terabytes.
• Fluency with scientific data semantics (studies, samples, donors, cell lines, assays) and the ability to translate scientist questions into data models.

Technologies

• AWS, S3, IAM, EC2, Batch, Lambda, Spark, EMR
• Python, SQL
• Nextflow, Airflow, Temporal, Step Functions, Redun
• CloudFormation, CDK, CI/CD, containers, observability
• Databricks, OpenMetadata, Snowflake, SageMaker, MLflow, Weights & Biases
• Feature stores
• LLM, MCP servers, tool APIs, Infrastructure as code

Benefits

• Healthcare coverage
• Annual incentive program
• Retirement benefits
• A broad range of other benefits

Preferred Qualifications

• Familiarity with genetics and genomics data.
• Microscopy or high-content screening data experience, including large binary object stores, transfer optimization, and image streaming for model training.
• Experience with Databricks, OpenMetadata, or a comparable catalog in production.
• Experience with Snowflake, especially governed access patterns over cloud object storage.
• ML infrastructure experience: SageMaker or Databricks ML; MLflow or Weights & Biases; feature stores.
• Experience serving data to LLM or agent clients (MCP servers, tool APIs).
• Experience leading data platform architecture as a senior or principal individual contributor, ideally in biotech, pharma R&D, or a research institute.

Location: Cambridge, MA (onsite)

Salary: USD 179,000 - 236,000 per year