Senior Data Platform Engineer: Databricks & Terraform Lead

Arrowhead Pharmaceuticals, Inc.; San Diego (CA); In loco; USD 150.000 - 210.000; Tempo pieno

Vantaggi offerti da questo lavoro

Excellent benefits

Competitive compensation

Descrizione del lavoro

Arrowhead Pharmaceuticals, Inc. is seeking a Senior Data Platform Engineer to own and optimize the Databricks data platform, including workspaces, clusters, Unity Catalog, and data pipelines.

You will implement Terraform-based infrastructure, secure access, and govern data quality across Bronze/Silver/Gold layers while enabling enterprise analytics through Genie dashboards. You will collaborate with data engineers, analysts, AI engineers, and cloud DevOps to ensure scalable, reliable pipelines

Competenze

• Bachelor's degree or equivalent practical experience.
• 5+ years in data or platform engineering with Databricks prod experience.
• Terraform IaC for Databricks infrastructure.
• Strong Python and SQL for data pipelines.
• Experience ingesting data from APIs, SFTP, batch and streaming sources.
• Cloud platforms (AWS, Azure) and data/storage services.
• Knowledge of Unity Catalog permissions and data governance.
• Ability to explain design decisions to non-technical stakeholders.

Mansioni

• Design, build, and manage Databricks environments (workspaces, clusters, Unity Catalog, SQL warehouses).
• Develop and maintain Terraform modules to provision Databricks infrastructure and cloud resources.
• Build data ingestion pipelines from internal systems and external providers into the lakehouse.
• Architect medallion data layers (bronze/silver/gold) with Delta Lake and governance.
• Collaborate with stakeholders to design Genie dashboards and enable self-service analytics.
• Implement platform security, access controls, and partner with data stewards.
• Work with CI/CD pipelines and infrastructure-as-code for the data platform.
• Document platform architecture, pipelines, and runbooks.
• Assist AI engineers with MCP connectors and data/tool integrations.

Conoscenze

Databricks

Terraform

Python

SQL

APIs/SFTP/Streaming

Unity Catalog

Delta Lake

CI/CD tooling

Cloud platforms (AWS/Azure)

Genie dashboards

Formazione

Bachelor's degree in Computer Science, Data Engineering, Information Systems, or related field

Strumenti

Databricks

Terraform

Delta Lake

Unity Catalog

GitHub Actions

Azure DevOps

Descrizione del lavoro

Arrowhead Pharmaceuticals, Inc. is seeking a Senior Data Platform Engineer to own and optimize the Databricks data platform, including workspaces, clusters, Unity Catalog, and data pipelines.

You will implement Terraform-based infrastructure, secure access, and govern data quality across Bronze/Silver/Gold layers while enabling enterprise analytics through Genie dashboards. You will collaborate with data engineers, analysts, AI engineers, and cloud DevOps to ensure scalable, reliable pipelines