Databricks Platform Lead: Terraform & Data Lakehouse Architect

Arrowhead Pharmaceuticals, Inc.; Madison (WI); On-site; USD 140,000 - 180,000; Full time

Benefits offered by this job

Competitive salary

Excellent benefits package

Job summary

Arrowhead Pharmaceuticals, Inc. is seeking a Senior Data Platform Engineer in Madison, WI to own Databricks environments end-to-end, including workspaces, clusters, Unity Catalog, and Delta Lake.

You will design data ingestion pipelines from internal and external sources and build CI/CD pipelines using Terraform for reliable production-grade infrastructure. The role partners with analytics teams to deliver Genie dashboards and enable self‑service data exploration, while maintaining platform

Qualifications

• Bachelor's degree in Computer Science, Data Engineering, Information Systems, or related field, or equivalent practical experience.
• 5+ years of experience in data engineering or platform engineering roles, including extensive hands-on experience administering and building on Databricks.
• Demonstrated experience managing Databricks infrastructure as code using Terraform.
• Strong proficiency in Python and SQL for building and optimizing data pipelines.
• Experience designing and building data ingestion pipelines from diverse internal and external data sources.
• Working knowledge of cloud platforms (AWS, Azure).
• Experience supporting BI or natural-language dashboarding tools (Databricks Genie).
• Experience implementing data governance, security, and access controls in Databricks (Unity Catalog).
• Ability to explain pipeline and dashboard design decisions to non-technical stakeholders.

Responsibilities

• Design, build, and manage Databricks environments (workspaces, clusters, Unity Catalog, SQL warehouses) supporting enterprise analytics and data science workloads.
• Develop and maintain Terraform modules and pipelines to provision, configure, and version-control all Databricks infrastructure and related cloud resources.
• Design and build data ingestion pipelines to bring in data from internal systems and external third-party data providers into the lakehouse.
• Architect and maintain medallion (bronze/silver/gold) data layers using Delta Lake, ensuring data quality, lineage, and governance via Unity Catalog.
• Partner with business and analytics end users to design, build, and support Genie dashboards and enable self-service, natural-language data exploration.
• Implement platform security, access controls, and partner with data stewards.
• Work with data engineers, analysts, data scientists, AI engineers, and cloud DevOps engineers to build and maintain CI/CD pipelines, automated tests, and Terraform infrastructure-as-code for the data platform.
• Document platform architecture, ingestion pipelines, and operational runbooks.
• Assist AI engineers by building and maintaining MCP connectors and other data/tool integrations.

Skills

Python

SQL

Databricks

Data ingestion

Delta Lake

Unity Catalog

Genie dashboards

Cloud platforms

CI/CD

Airflow

dbt

Education

Bachelor's degree in CS/Info Systems/Data Eng

Tools

Terraform

Databricks

GitHub Actions

Azure DevOps

GitLab CI

Airflow

dbt

Job description

Arrowhead Pharmaceuticals, Inc. is seeking a Senior Data Platform Engineer in Madison, WI to own Databricks environments end-to-end, including workspaces, clusters, Unity Catalog, and Delta Lake.

You will design data ingestion pipelines from internal and external sources and build CI/CD pipelines using Terraform for reliable production-grade infrastructure. The role partners with analytics teams to deliver Genie dashboards and enable self‑service data exploration, while maintaining platform