Senior Staff Data Engineer — Large-Scale Bio AI Pipelines

BioHub; New York (NY); Hybrid; USD 270,000 - 338,000; Full time

Benefits offered by this job

Relocation assistance

401(k) match

Volunteer time off

Job summary

Biohub is seeking a Senior Staff Data Engineer to design data pipelines and infrastructure for AI research, ingesting public and internal data and delivering training datasets. You will transform diverse biological data into AI-ready formats and collaborate with researchers to accelerate discovery.

Based in a hybrid setting with NY and Redwood City offices, you will own data strategy, quality, and scale, ensuring reliability and observability as data volumes grow into the petabyte range.

Qualifications

• 8+ years of experience building reliable data systems at petabyte scale.
• Strong software engineering fundamentals.
• Experience deploying distributed computing frameworks like Databricks, Spark or Ray.
• Experience building data platforms using infrastructure as code (Terraform, CDK).
• Experience with cloud infrastructure (AWS preferred) and on-premise infra.
• Comfort with ambiguity and evolving requirements.
• Interest in AI-native development practices and tooling.
• Nice to have: background in computational biology, bioinformatics, or genomics/imaging data formats.

Responsibilities

• Design and build data pipelines that process genomic and imaging data at petabyte scale.
• Solve performance and bandwidth challenges with creative engineering.
• Build agent-based systems for automated dataset curation, quality control, and workflow generation.
• Create tooling for data cataloging and registration that makes datasets discoverable and accessible.
• Collaborate with AI Research teams to translate model requirements into data specs and integrate data.
• Improve pipeline reliability and observability toward 99%+ success rates.

Skills

Software engineering fundamentals

Distributed computing

Infrastructure as code

Cloud & on-prem infrastructure

Ambiguity tolerance

AI-native development

Genomics/biological data awareness

Tools

Terraform

CDK

Databricks

Spark

Ray

Job description

Biohub is seeking a Senior Staff Data Engineer to design data pipelines and infrastructure for AI research, ingesting public and internal data and delivering training datasets. You will transform diverse biological data into AI-ready formats and collaborate with researchers to accelerate discovery.

Based in a hybrid setting with NY and Redwood City offices, you will own data strategy, quality, and scale, ensuring reliability and observability as data volumes grow into the petabyte range.