Senior Data Platform Engineer — Azure & Databricks

Kestra; Tempe (AZ); On-site; USD 140,000 - 190,000; Full time

Benefits offered by this job

Competitive pay

401(k) and health insurance

Growth opportunities

Job summary

Kestra Holdings seeks a seasoned Databricks Data Engineer in Tempe to design and optimize large-scale data pipelines on the Databricks Lakehouse, with Azure cloud services, Delta Lake, and Unity Catalog for governance.

You will build scalable ingestion pipelines, implement CI/CD automation, and collaborate with cross-functional teams to advance data models and secure data sharing for BI and analytics.

Qualifications

• 8+ years designing scalable data pipelines in modern data warehousing environments.
• Advanced proficiency in SQL and Python with ETL/ELT expertise.
• Experience with Azure, Snowflake, and Databricks in enterprise settings.

Responsibilities

• Design, build, and optimize large-scale data pipelines on the Databricks Lakehouse platform.
• Modernize the Azure-based data ecosystem with cloud architecture, governance, and CI/CD automation.
• Utilize Apache Airflow for orchestration and workflow automation.
• Work with regulated datasets ensuring compliance and governance.
• Write optimized SQL/T-SQL queries and curate datasets for reporting.

Skills

Data engineering

ETL/ELT frameworks

SQL

Python

Tools

Azure

Snowflake

Databricks

Airflow

Job description

Kestra Holdings seeks a seasoned Databricks Data Engineer in Tempe to design and optimize large-scale data pipelines on the Databricks Lakehouse, with Azure cloud services, Delta Lake, and Unity Catalog for governance.

You will build scalable ingestion pipelines, implement CI/CD automation, and collaborate with cross-functional teams to advance data models and secure data sharing for BI and analytics.