Data Engineer for AI-Driven Drug Discovery

BioSpace; San Francisco (CA); Hybrid; USD 158,000 - 231,000; Full time

Benefits offered by this job

Company bonus

401(k)

Job summary

Lilly, in partnership with NVIDIA, seeks a Data Engineer to build and maintain data platforms powering AI-driven drug discovery. You will ingest, transform, and deliver chemical, biological, and experimental data for ML workflows, partnering with AI Scientists and lab researchers to ensure data accuracy and accessibility.

You will design scalable pipelines, ensure data versioning and lineage, and optimize performance across cloud-based compute environments.

Qualifications

• BS/BA in CS, data science, engineering or related field.
• 5+ years building production data systems.
• Experience with scalable data pipelines and data quality.

Responsibilities

• Build and maintain data platforms powering AI-driven research.
• Develop scalable pipelines ingesting, transforming, and delivering data for ML workflows.
• Collaborate with AI scientists and lab researchers to ensure data accuracy.
• Ensure data is traceable, versioned, and reproducible for model training.
• Optimize data flows for performance and scalability.
• Ensure upstream changes do not corrupt downstream training runs.

Skills

Python

SQL

Data modeling

Distributed data processing

Cloud platforms

CI/CD

Collaboration

Education

Bachelor’s degree in CS/Data Science/Engineering

Tools

Spark

Airflow

Dagster

PostgreSQL

Python ecosystem

Job description

Lilly, in partnership with NVIDIA, seeks a Data Engineer to build and maintain data platforms powering AI-driven drug discovery. You will ingest, transform, and deliver chemical, biological, and experimental data for ML workflows, partnering with AI Scientists and lab researchers to ensure data accuracy and accessibility.

You will design scalable pipelines, ensure data versioning and lineage, and optimize performance across cloud-based compute environments.