Azure Databricks Platform Engineer: Governance & Ops

Lumen Resources; New York (NY); On-site; USD 160,000 - 210,000; Full time

Job summary

Lumen Resources is seeking an Azure Databricks Platform Engineer to own the administration, governance, security, and operational health of an enterprise Databricks environment.

This role focuses on platform health, Unity Catalog governance, access controls, workspaces, compute, production jobs, and cost-efficient operations across data, analytics, and AI workloads.

The position emphasizes platform administration over traditional data engineering, with automation and IaC experience beneficial.

Qualifications

• Hands-on Databricks administration in a production enterprise environment.
• Strong Unity Catalog governance, permissions, schemas, and access controls.
• Experience managing Databricks workspaces, clusters/compute, jobs, and platform configurations.

Responsibilities

• Administer Databricks environments across development to production workloads.
• Own day-to-day platform health: workloads, compute, availability, and operational activity.
• Administer Unity Catalog and scalable governance for data, permissions, and access.
• Manage users, groups, roles, and access controls with data protection.
• Troubleshoot production Databricks issues and platform connectivity.
• Configure and optimize clusters and compute resources for workloads and cost.
• Establish consistent admin and security standards across environments.
• Support controlled promotion of platform configurations and security standards.

Skills

Databricks administration

Unity Catalog

Workspace management

Cluster optimization

Security governance

Cost optimization

Azure cloud & cost sensing

Tools

Terraform

CI/CD

GitHub

Azure DevOps

Power BI

Microsoft Purview

Job description

Lumen Resources is seeking an Azure Databricks Platform Engineer to own the administration, governance, security, and operational health of an enterprise Databricks environment.

This role focuses on platform health, Unity Catalog governance, access controls, workspaces, compute, production jobs, and cost-efficient operations across data, analytics, and AI workloads.

The position emphasizes platform administration over traditional data engineering, with automation and IaC experience beneficial.