Senior Data Engineer: Scientific Pipelines & AI-Ready Data

Lilasciences; San Francisco (CA); On-site; USD 144,000 - 240,000; Full time

Benefits offered by this job

Medical insurance

Life & disability insurance

Flexible PTO

Parental leave

Educational assistance

Commuter benefits

Lunch program

Job summary

Lila Sciences in San Francisco, CA, is seeking a Data Engineer to build ETL pipelines and data models for its scientific data platform, collaborating with AI researchers and experimentalists.

You will transform raw instrument outputs into validated datasets, implement quality checks, and create reusable analytics functions, with a focus on scalable, reliable data flows and governance.

Qualifications

• 2–6 years of experience in data engineering, bioinformatics, cheminformatics, or computational science.
• Strong Python skills, including typed, tested, production-quality code.
• Strong SQL skills, especially with Postgres or similar relational databases.
• Experience building ETL pipelines, data models, and reusable data transformations.
• Data science foundation, including statistics and pandas, NumPy, or similar tools.
• Experience translating noisy scientific measurements into accurate, validated datasets.
• Workflow orchestration experience, ideally Flyte, Airflow, Prefect, Dagster, or Nextflow.
• Active use of AI coding tools in day-to-day engineering work.

Responsibilities

• Design pipelines that turn raw lab output into analysis-ready scientific data.
• Model heterogeneous data from bio, chemistry, and materials instruments.
• Build validation checks, schema-evolution gates, and data quality workflows.
• Develop reusable analysis functions for scientific and AI research workflows.
• Improve automation and observability across instrument-to-result data flows.
• Build canonical datasets that scientists and AI researchers can trust.
• Use AI coding tools to accelerate pipeline development and team velocity.

Skills

Python

SQL

ETL pipelines

Data modeling

Pandas/NumPy

Workflow orchestration

Tools

Postgres

Airflow

Flyte

Prefect

Dagster

DuckDB

Job description

Lila Sciences in San Francisco, CA, is seeking a Data Engineer to build ETL pipelines and data models for its scientific data platform, collaborating with AI researchers and experimentalists.

You will transform raw instrument outputs into validated datasets, implement quality checks, and create reusable analytics functions, with a focus on scalable, reliable data flows and governance.