Senior Databricks Engineer — Data Pipelines & Governance

Remotedxb; Dubai; On-site; AED 250,000 - 400,000; Full time

Job summary

Geisinger is seeking a seasoned Databricks and data engineering lead to manage workspaces, clusters, jobs and security, ensuring platform health across our cloud ecosystem. You will build data pipelines with PySpark and T-SQL, and create automation tooling using Python, Bash and PowerShell, while enforcing data governance and security best practices.

You'll translate ideas into architectures and actionable tasks in Azure DevOps, participate in 24x7 on-call rotations for Databricks and SQL

Qualifications

• Extensive Databricks administration experience.
• Proficient in Python, Bash and PowerShell.
• Experience with PySpark and T-SQL data processing.
• Solid knowledge of data governance and security.
• Minimum 12 years of relevant experience.
• At least 3 years of people, process, or project leadership.

Responsibilities

• Manage Databricks workspaces, clusters, jobs, and security.
• Build data pipelines and transformations with PySpark and T-SQL.
• Create automation tooling using Python, Bash, and PowerShell.
• Implement data governance and access controls.
• Contribute to design architectures and task planning in Azure DevOps.
• Support Databricks and SQL Server in a 24x7 on-call rotation.
• Produce documentation including design specs and test plans.

Skills

Databricks admin

Python

Bash

PowerShell

PySpark

T-SQL

Data governance

Information security

People management

Project management

Tools

Terraform

Azure DevOps

GitHub

AWS Glue

AWS Lambda

MS SQL Server

Job description

Geisinger is seeking a seasoned Databricks and data engineering lead to manage workspaces, clusters, jobs and security, ensuring platform health across our cloud ecosystem. You will build data pipelines with PySpark and T-SQL, and create automation tooling using Python, Bash and PowerShell, while enforcing data governance and security best practices.

You'll translate ideas into architectures and actionable tasks in Azure DevOps, participate in 24x7 on-call rotations for Databricks and SQL