Senior Data Platform Engineer

Arrowhead Pharmaceuticals, Inc.; Madison (WI); On-site; USD 140,000 - 180,000; Full time

Benefits offered by this job

Competitive salary

Excellent benefits package

Job summary

Arrowhead Pharmaceuticals, Inc. is seeking a Senior Data Platform Engineer in Madison, WI to own Databricks environments end-to-end, including workspaces, clusters, Unity Catalog, and Delta Lake.

You will design data ingestion pipelines from internal and external sources and build CI/CD pipelines using Terraform for reliable production-grade infrastructure. The role partners with analytics teams to deliver Genie dashboards and enable self‑service data exploration, while maintaining platform

Qualifications

• Bachelor's degree in Computer Science, Data Engineering, Information Systems, or related field, or equivalent practical experience.
• 5+ years of experience in data engineering or platform engineering roles, including extensive hands-on experience administering and building on Databricks.
• Demonstrated experience managing Databricks infrastructure as code using Terraform.
• Strong proficiency in Python and SQL for building and optimizing data pipelines.
• Experience designing and building data ingestion pipelines from diverse internal and external data sources.
• Working knowledge of cloud platforms (AWS, Azure).
• Experience supporting BI or natural-language dashboarding tools (Databricks Genie).
• Experience implementing data governance, security, and access controls in Databricks (Unity Catalog).
• Ability to explain pipeline and dashboard design decisions to non-technical stakeholders.

Responsibilities

• Design, build, and manage Databricks environments (workspaces, clusters, Unity Catalog, SQL warehouses) supporting enterprise analytics and data science workloads.
• Develop and maintain Terraform modules and pipelines to provision, configure, and version-control all Databricks infrastructure and related cloud resources.
• Design and build data ingestion pipelines to bring in data from internal systems and external third-party data providers into the lakehouse.
• Architect and maintain medallion (bronze/silver/gold) data layers using Delta Lake, ensuring data quality, lineage, and governance via Unity Catalog.
• Partner with business and analytics end users to design, build, and support Genie dashboards and enable self-service, natural-language data exploration.
• Implement platform security, access controls, and partner with data stewards.
• Work with data engineers, analysts, data scientists, AI engineers, and cloud DevOps engineers to build and maintain CI/CD pipelines, automated tests, and Terraform infrastructure-as-code for the data platform.
• Document platform architecture, ingestion pipelines, and operational runbooks.
• Assist AI engineers by building and maintaining MCP connectors and other data/tool integrations.

Skills

Python

SQL

Databricks

Data ingestion

Delta Lake

Unity Catalog

Genie dashboards

Cloud platforms

CI/CD

Airflow

dbt

Education

Bachelor's degree in CS/Info Systems/Data Eng

Tools

Terraform

Databricks

GitHub Actions

Azure DevOps

GitLab CI

Airflow

dbt

Job description

The Position

The Senior Data Platform Engineer is responsible for designing, building, and managing the organization's Databricks environments, with end-to-end ownership of the data platform's infrastructure, security, and performance. This role leads the ingestion of data from internal systems and external third‑party providers into the lakehouse, partners with business and analytics stakeholders to build and support Genie dashboards, and manages the Databricks platform (workspaces, clusters, Unity Catalog, jobs) as infrastructure‑as‑code using Terraform. The ideal candidate has hands‑on experience administering Databricks in production, writing Terraform for infrastructure changes, and building data pipelines that hold up at enterprise scale.

Responsibilities

• Design, build, and manage Databricks environments (workspaces, clusters, Unity Catalog, SQL warehouses) supporting enterprise analytics and data science workloads


• Develop and maintain Terraform modules and pipelines to provision, configure, and version‑control all Databricks infrastructure and related cloud resources


• Design and build data ingestion pipelines to bring in data from internal systems and external third‑party data providers (APIs, SFTP, batch files, streaming sources) into the lakehouse.


• Architect and maintain medallion (bronze/silver/gold) data layers using Delta Lake, ensuring data quality, lineage, and governance via Unity Catalog


• Partner with business and analytics end users to design, build, and support Genie dashboards and enable self‑service, natural‑language data exploration


• Implement platform security, access controls, and partner with data stewards.


• Work with data engineers, analysts, data scientists, AI engineers, and cloud DevOps engineers to build and maintain CI/CD pipelines, automated tests, and Terraform infrastructure‑as‑code for the data platform


• Document platform architecture, ingestion pipelines, and operational runbooks


• Assist AI engineers by building and maintaining MCP (Model Context Protocol) connectors and other data/tool integrations that power agentic workflows

Requirements

• Bachelor's degree in Computer Science, Data Engineering, Information Systems, or related field, or equivalent practical experience


• 5+ years of experience in data engineering or platform engineering roles, including extensive hands‑on experience administering and building on Databricks (clusters, jobs, Unity Catalog, Delta Lake, SQL warehouses)


• Demonstrated experience managing Databricks infrastructure as code using Terraform


• Strong proficiency in Python and SQL for building and optimizing data pipelines


• Experience designing and building data ingestion pipelines from diverse internal and external data sources (APIs, SFTP, third‑party vendor feeds, streaming)


• Working knowledge of cloud platforms (AWS, Azure ) and associated data/storage services


• Experience supporting business intelligence or natural‑language dashboarding tools (e.g., Databricks Genie )


• Experience implementing data governance, security, and access controls in Databricks (e.g., Unity Catalog permissions, row/column‑level security, service principals, secrets management)


• Ability to explain pipeline and dashboard design decisions to non‑technical stakeholders, e.g., when scoping Genie dashboard requirements with business users

Preferred

• Databricks certifications (e.g., Databricks Certified Data Engineer Professional, Databricks Certified Associate/Professional Platform Administrator)


• Experience with Databricks Genie and building natural‑language / self‑service analytics experiences for business users


• Experience integrating data from external data providers or vendor APIs at enterprise scale


• Familiarity with CI/CD tooling (e.g., GitHub Actions, Azure DevOps, GitLab CI) for automated deployment of Terraform‑managed infrastructure


• Experience with data orchestration tools (e.g., Databricks Workflows, Airflow, dbt)


• Experience building and supporting agentic workloads on Databricks, including managing model/tool access and governance via the Unity Catalog AI Gateway


• Expertise in pharma or pharma‑related data (e.g., clinical, R&D, regulatory, or commercial pharma data) and the compliance considerations that come with it

Arrowhead provides competitive salaries and an excellent benefit package.

Candidates must have current, valid authorization to work in the country where this role is located.