Senior Data Engineer

Investigo; Palo Alto (CA); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

401k

Equity

On-site lunch

Job summary

A leading generative AI company is seeking a Senior Data Engineer to design and maintain data platforms that facilitate AI solutions in healthcare. This role involves building data pipelines, ensuring compliance with data security standards, and collaborating across teams to optimize performance in complex systems. Candidates should have substantial experience in software or data engineering, particularly with data infrastructure, and proficiency in Python and SQL. Ideal for those passionate about advancing AI in a safety-first environment.

Qualifications

• 5+ years of experience in software or data engineering.
• Familiarity with orchestration tools and infrastructure-as-code.
• Strong understanding of data security and compliance frameworks.

Responsibilities

• Build and operate data platforms and pipelines for training and analytics.
• Own data governance and access control with HIPAA-grade compliance.
• Ensure reliability and cost efficiency across storage and data quality checks.

Skills

Python

SQL

Data governance

ETL/ELT pipelines

Data security

Collaboration skills

Tools

Airflow

Prefect

dbt

Snowflake

Databricks

BigQuery

Redshift

Job description

Senior Data Engineer

We are looking for a Senior Data Engineer to work with a leading generative AI company in healthcare.

Location: Palo Alto, CA office. This role is expected to be in our Palo Alto office five days a week. Benefits include 401k, equity, and on-site lunch.

Why Join Our Team

• Reinvent healthcare with AI that puts safety first.
• Work with the people shaping the future.
• Backed by the world’s leading healthcare and AI investors.
• Build alongside the best in healthcare and AI.

What You'll Do

• Build and operate data platforms and pipelines (batch/stream) that feed training, RAG, evaluation, and analytics using tools like Prefect, dbt, Airflow, Spark, and cloud data warehouses (Snowflake/BigQuery/Redshift).
• Own data governance and access control: implement HIPAA‑grade permissioning, lineage, audit logging, DLP, manage IAM, roles, and policy‑as‑code.
• Ensure reliability, observability, and cost efficiency across storage (S3/GCS), warehouses, ETL/ELT, SLAs/SLOs, data quality checks, monitoring, and disaster recovery.
• Enable self‑service analytics via curated models and semantic layers; mentor engineers on best practices in schema design, SQL performance, and data lifecycle. Partner with ML/Research to provision high‑quality datasets, feature stores, and labeling/eval corpora with reproducibility (versioning, metadata, data contracts).

What You Bring

Must‑Have Qualifications

• 5+ years of software or data engineering experience, with 3+ years building data infrastructure, ETL/ELT pipelines, or distributed data systems.
• Deep experience with Python and at least one cloud data platform (Snowflake, Databricks, BigQuery, Redshift, or equivalent).
• Familiarity with orchestration tools (Airflow, Prefect, dbt) and infrastructure‑as‑code (Terraform, CloudFormation).
• Strong understanding of data security, access control, and compliance frameworks (HIPAA, SOC 2, GDPR, or similar).
• Proficiency with SQL and experience optimizing query performance and storage design.
• Excellent problem‑solving and collaboration skills — able to work across engineering, ML, and clinical teams.
• Comfortable navigating trade‑offs between performance, cost, and maintainability in complex systems.

Nice‑to‑Have

• Experience supporting ML pipelines, feature stores, or model training datasets.
• Familiarity with real‑time streaming systems (Kafka, Kinesis) or large‑scale unstructured data storage (S3, GCS).
• Background in data reliability engineering, data quality monitoring, or governance automation.
• Experience in healthcare, safety‑critical systems, or regulated environments.

If you’re passionate about building data systems that power safe, real‑world AI, we’d love to hear from you. Apply today and take the next step in your career!