Senior Data Pipeline Engineer/Developer

Gene by Gene; Houston (TX); On-site; USD 110,000 - 170,000; Full time

Job summary

Gene by Gene is seeking a Senior Data Pipeline Engineer/Developer to design, build, and own data pipelines moving genomic and operational data from lab instruments through analytics, products, and clinical reporting. This senior IC role sets technical direction for the data platform, delivering reliable pipelines at scale in a regulated environment.

You will work in Python-first stack, guide engineering teams, ensure data quality, lineage, observability, and compliance with HIPAA, GxP, and

Qualifications

• 8+ years of professional software or data engineering experience.
• 4+ years building and operating production data pipelines at scale.
• 3+ years of production cloud experience (AWS preferred).
• Experience in regulated environments with HIPAA or GxP audits.
• Bachelor's degree in CS, Bioinformatics, or related field; advanced degree preferred.

Responsibilities

• Builds and operates production ETL and ELT for high-volume genomic data as well as operational and business data.
• Integrates data across LIMS, lab instruments, internal applications, and third‑party sources.
• Develops across the stack in Python (data services, internal APIs, and supporting application code).
• Sets technical direction for batch and streaming data pipelines; leads adoption.
• Models and tunes data stores (SQL Server and PostgreSQL, cloud warehouse) for performance.
• Defines and enforces engineering standards for testing, CI/CD, IaC, code reviews, and architecture decisions.
• Owns data quality, lineage, and observability with SLAs and cost controls.
• Partners with security/compliance on data privacy and regulatory requirements across the lifecycle.

Skills

Python

SQL

Data pipelines

Docker & Kubernetes

Airflow/Orchestrators

Cloud AWS

Data quality & testing

Communication

Genomics data

Education

Bachelor's degree in CS/Bioinformatics

Advanced degree preferred

Tools

Argo Workflows

Airflow

Dagster

Prefect

Nextflow

WDL/Cromwell

Docker

Kubernetes

Snowflake

dbt

Terraform

CloudFormation

Pulumi

S3

Redshift

Job description

Our mission is to build a healthier and more connected world with precision health and genealogy services. We empower individuals with actionable insights into their genetic makeup, fostering a deeper understanding of their ancestry, health, and wellness. By integrating the experience of Gene by Gene Laboratory Services, FamilyTreeDNA genealogy, and myDNA reporting services, we strive to deliver cutting‑edge genetic testing and personalized solutions that inspire informed decisions and enhance quality of life. Our team is dedicated to advancing the field of genomics through innovation, research, and a commitment to excellence.

Our Values

All employees are expected to demonstrate our values of Innovate , One Team , and Integrity when carrying out the accountabilities and responsibilities of their role. This is how we show up every day for ourselves, our colleagues and our customers and strategic partners to deliver our vision and strategic goals.

Position Overview

We are seeking a Senior Data Pipeline Engineer/Developer to design, build, and own the data pipelines that move genomic and operational data from lab instruments and LIMS through to analytics, products, and clinical reporting. In this senior individual‑contributor role you will set technical direction for our data platform, build production pipelines that are reliable and reproducible at scale, and provide architectural and code‑level guidance to existing engineering teams. You will own data quality, lineage, and observability end to end, and operate in a regulated environment where reproducibility and auditability are non‑negotiable. This is a Python‑first, full‑stack engineering role. This is a contractor position, slated for a 12‑month term.

Accountabilities and Responsibilities

• Builds and operates production ETL and ELT for high-volume genomic data as well as operational and business data.
• Integrates data across LIMS, lab instruments, internal applications, and third‑party sources through robust, well‑tested interfaces.
• Develops across the stack in Python (data services, internal APIs, and supporting application code) and provides architectural and code‑level guidance to engineering teams on data‑layer integration.
• Architecture & Technical Leadership
• Sets technical direction for batch and streaming data pipelines by evaluating frameworks, orchestration, storage, and processing patterns, making recommendations, and leading adoption.
• Models and tunes data stores (Microsoft SQL Server and PostgreSQL, plus a cloud warehouse or lake) for performance and scale.
• Defines and enforces engineering standards for testing, CI/CD, infrastructure as code, code review, and architecture decision records.
• Data Quality & Pipeline Observability
• Owns data quality, lineage, and observability, including freshness, completeness, schema‑drift detection, cost‑per‑job, and SLAs.
• Builds pipelines for reproducibility and audit‑readiness, incorporating versioned data and code, lineage, decision logging, access controls, and evidence collection.
• Partners with security and compliance on data privacy, PII and PHI handling, and regulatory requirements across the data lifecycle.
• Regulatory Compliance and Security Governance
• Apply deep understanding of CAP/CLIA, HIPAA, GDPR, and GxP regulations specifically to data pipeline architecture.
• Oversee protected health information (PHI) handling, data lineage, retention, and comprehensive audit logging.
• Produce and maintain the critical operational evidence required to carry the data platform successfully through compliance audits.
• Adhere to strict data‑governance controls necessary for securely handling sensitive genomic data across global regions.
• Enforce least‑privilege access and ensure zero data egress to personal or unapproved infrastructure.
• Utilize exclusively de‑identified or synthetic data within development environments.
• Maintain strict compliance with international data‑residency and localization requirements.

Position Requirements

• Skills and Knowledge
• Strong SQL proficiency on Microsoft SQL Server and PostgreSQL, including schema design, query tuning, and performance troubleshooting.
• Strong Python proficiency across the stack (data pipelines, backend services, and APIs). This is a Python‑first role.
• Production experience with a workflow orchestrator (Argo Workflows, Step Functions, Prefect, Dagster, Airflow, or similar).
• Production experience with containers (Docker) and Kubernetes, which the lead orchestrator (Argo Workflows) runs on.
• Strong testing discipline, including unit, integration, and data‑quality or contract tests.
• Excellent written and verbal communication, including the ability to explain data tradeoffs to technical and compliance stakeholders.
• Genomics or NGS data formats and handling (FASTQ, BAM/CRAM, VCF).
• Bioinformatics workflow engines (Nextflow, WDL/Cromwell, or Snakemake).
• AWS data stack proficiency (S3, Glue, EMR, Batch, Lambda, Redshift) and/or AWS HealthOmics.
• Distributed processing and streaming frameworks (Spark, Kafka, Kinesis).
• Data warehouse and transformation tooling (Snowflake, dbt).
• Infrastructure as code practices (Terraform, CDK, CloudFormation, or Pulumi).
• Familiarity with .NET (C#) and/or C++ as supporting languages for integrating with existing services.
• Knowledge of LIMS integration and on‑premises plus hybrid data architectures.
• Experience
• 8+ years of professional software or data engineering experience.
• 4+ years building and operating production data pipelines at scale.
• 3+ years of production cloud experience (AWS preferred).
• Demonstrable experience working in regulated environments. Compliance is a hard requirement for this role.
• Experience supporting HIPAA or GxP audits.
• Education
• Bachelor's degree in Computer Science, Bioinformatics, or a related field, or equivalent professional experience.
• An advanced degree in a quantitative field is preferred.

Why Join Us

At Gene by Gene, you’ll join a mission‑driven team advancing the science of genetics and discovery. You’ll have the opportunity to shape meaningful campaigns, tell compelling brand stories, and collaborate with talented professionals who share your passion for creativity, curiosity, and impact.