Neuroscience Data Platform Engineer

SLAC; Palo Alto (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

SLAC invites applications for a Data Engineer (Software Developer 2) to design, build, and maintain a data platform supporting Enigma’s experimental workflows. You will own end-to-end data pipelines, implement durable data models, and partner with researchers to enable reliable data processing and governance across cloud storage, processing, and downstream stores.

The role emphasizes productionizing research code, observability, and cross-functional collaboration in a fast-paced lab environment

Qualifications

• Strong proficiency in Python and SQL.
• Experience with relational and NoSQL databases, as well as object storage (S3, MinIO, Ceph).
• Experience designing, building, and operating production ETL/ELT pipelines.
• Experience designing durable data models and schemas for complex, evolving datasets.
• Experience building high-throughput ingestion systems and optimizing data movement across storage, compute, and network boundaries.
• Experience with Docker and container orchestration platforms such as Kubernetes.
• Strong communication skills and experience working in cross-functional teams.
• Experience designing APIs or service interfaces used by multiple teams.
• Experience with scientific computing and large-scale neuroscience or other scientific data.
• Workflow orchestration experience with Airflow, Dagster, Prefect, or similar tools.
• Experience in building pipelines with and for autonomous, agentic AI systems.
• Experience with performance-critical compiled or systems languages (C, C++, or Rust).
• Familiarity with data provenance and lineage tracking, metadata catalogs, or dataset versioning.

Responsibilities

• Design, build, and operate ETL/ELT and high-throughput ingestion pipelines connecting acquisition software, processing code, metadata registries, object storage, and databases.
• Own the data layer end-to-end: schema design, indexing, query optimization, migrations, backups, and durable data models that hold up over years of scientific work.
• Improve reliability, scalability, observability, and performance across the data stack, and lead troubleshooting/incident response when things break.
• Build tools with and to help researchers and our in-house AI agents discover datasets, inspect processing state, and safely diagnose or rerun failed jobs.
• Productionize research code - testing, packaging, deployment, monitoring, documentation.
• Partner with data acquisition, infrastructure engineering, scientific analysis and AI modeling teams.

Skills

Python

SQL

Data modeling

Data warehousing

Tools

Docker

Kubernetes

Airflow

Dagster

Prefect

S3

MinIO

Ceph

Job description

SLAC invites applications for a Data Engineer (Software Developer 2) to design, build, and maintain a data platform supporting Enigma’s experimental workflows. You will own end-to-end data pipelines, implement durable data models, and partner with researchers to enable reliable data processing and governance across cloud storage, processing, and downstream stores.

The role emphasizes productionizing research code, observability, and cross-functional collaboration in a fast-paced lab environment