Senior Data Platform Engineer - Scale & Enrich Datasets

Doist; Somerville (MA); Hybrid; USD 150,000 - 210,000; Full time

Benefits offered by this job

Health insurance

Dental insurance

Vision insurance

Disability insurance

Life insurance

401k with company match

Commuter benefits

Parking

Unlimited time away

Education/conference support

Job summary

Matterworks is seeking a Senior Software Engineer to build the connective tissue of our data platform, designing and scaling systems that enrich data from raw samples into usable datasets with biological context. You will work daily with ML researchers, scientists, and product teams to serve data for research and product development.

You will own data contracts, enable scalable data processing, and drive production-quality checks, cost-aware delivery, and robust interfaces for cross-disciplinary

Qualifications

• Significant professional experience building production data systems and pipelines.
• Proficient in Python and SQL for large-scale data processing.
• Proficient in Kubernetes-native batch orchestration and modern data lake technologies.
• Demonstrated experience designing stable identifiers for a large, changing corpus.
• Experience putting an LLM or agent component into a production data path.
• Daily use of AI coding tools with caution on production data correctness.
• ownership of work through to running, validated systems.
• Comfort with messy scientific formats and toolchains.
• Autonomy and eagerness to learn in an early-stage startup.

Responsibilities

• Build and Scale Data Contracts: Own the pipelines and system others consume from.
• Serve and Cost at Scale: Build and scale systems to acquire, store and serve data efficiently.
• Labels and Enrichment: Turn raw data into usable datasets with consistent schemas and metadata.
• Quality Gates: Automate quality checks that ensure progress with no regression.
• Interfaces People Use: Own surfaces called by SDKs and tools exposing platform capabilities.
• Operations and Data Rights: Ensure SLAs, provenance, and secure data processing.

Skills

Python

SQL

Kubernetes

Data pipelines

Large-scale data

Argo Workflows

Metaflow

EKS

Glue

Athena

Iceberg

Parquet

DuckDB

Terraform

Tools

Argo Workflows

Metaflow

EKS

Glue

Athena

Apache Iceberg

Parquet

DuckDB

Terraform

Airflow

Dagster

Job description

Matterworks is seeking a Senior Software Engineer to build the connective tissue of our data platform, designing and scaling systems that enrich data from raw samples into usable datasets with biological context. You will work daily with ML researchers, scientists, and product teams to serve data for research and product development.

You will own data contracts, enable scalable data processing, and drive production-quality checks, cost-aware delivery, and robust interfaces for cross-disciplinary