Data Engineer — Scalable Pipelines & Data Governance

Scorpion Therapeutics; Indianapolis (IN); On-site; USD 120,000 - 180,000; Full time

Job summary

Scorpion Therapeutics is seeking a data engineer to design, build, and optimize scalable data pipelines across the CE data ecosystem. You will work with Databricks, PySpark, Python, SQL, and Delta Lake to ingest, transform, and store data in data warehouses and data lakes.

You will implement the Bronze–Silver–Gold medallion architecture, establish data governance, and enable secure access to PHI while collaborating with analytics and product teams in an Agile environment.

Qualifications

• Bachelor’s degree in CS/Engineering/IS or related quantitative field.
• 5+ years data engineering/ETL development.
• Proficient in SQL and at least one language (Python/Java/Databricks).
• Experience with cloud data platforms (Databricks, AWS/Azure/GCP) and services (S3, Redshift, Snowflake, ADLS, BigQuery).
• Proficient with Git-based CI/CD workflows (GitHub Actions or equivalent).

Responsibilities

• Design, develop, and optimize scalable data pipelines using Databricks and Python.
• Implement canonical data models across medallion architecture Bronze → Silver → Gold.
• Evaluate/apply Databricks capabilities for performance, cost, and scalability.
• Implement and maintain ELT/ETL workflows (Databricks Workflows, Auto Loader, Structured Streaming, Delta Live Tables).
• Build CI/CD pipelines for CE data and artifacts.
• Automate data ingestion and product creation to reduce manual maintenance and onboarding.

Skills

Data engineering

SQL

Python / Databricks

Cloud platforms

Education

Bachelor’s degree in CS/Engineering/IS or related field

Tools

Databricks

AWS/Azure/GCP

S3 / Redshift

Snowflake / BigQuery

Job description

Scorpion Therapeutics is seeking a data engineer to design, build, and optimize scalable data pipelines across the CE data ecosystem. You will work with Databricks, PySpark, Python, SQL, and Delta Lake to ingest, transform, and store data in data warehouses and data lakes.

You will implement the Bronze–Silver–Gold medallion architecture, establish data governance, and enable secure access to PHI while collaborating with analytics and product teams in an Agile environment.