Databricks Platform Engineer: IAM, Automation & Governance

Black Rock Groups; Maine; On-site; USD 120,000 - 180,000; Full time

Job summary

Black Rock Groups is seeking a BI Admin Databricks specialist to manage and optimize our Databricks workspace across development, testing, and production environments. The role emphasizes IAM, governance, and incident response within Azure Cloud.

You will configure cluster policies, manage entitlements, monitor performance, and contribute to CI/CD and IaC practices to ensure secure and scalable data pipelines.

Qualifications

• Experience with Azure Cloud and IAM controls.
• Experience with Databricks workspace admin and governance.
• Strong troubleshooting and cross-functional communication.
• Familiarity with monitoring, logging, and incident response.
• Automation scripting using Python, PowerShell, Bash, or Terraform.

Responsibilities

• Administer and support Databricks workspace across dev, test, and prod.
• Manage user provisioning, RBAC, service principals, groups, entitlements.
• Configure cluster policies, instance pools, job compute, and resources.
• Monitor health, performance, usage trends, and incidents.
• Implement secrets management, data access, encryption, and audit logging.
• Support Unity Catalog and data governance at workspace level.
• Develop automation for workspace setup, policy enforcement, monitoring.
• Support CI/CD and IaC for Databricks deployments.
• Troubleshoot platform, connectivity, identity federation, and job issues.
• Create operational docs, runbooks, support procedures.
• Understand Spark workflows, Delta Lake, and scheduling.

Skills

Azure Cloud

IAM & RBAC

Databricks admin

Networking concepts

Python/Automation

Monitoring & Alerts

CI/CD / IaC

Troubleshooting

Documentation

Collaboration

Tools

Terraform

PowerShell

Bash

Job description

Black Rock Groups is seeking a BI Admin Databricks specialist to manage and optimize our Databricks workspace across development, testing, and production environments. The role emphasizes IAM, governance, and incident response within Azure Cloud.

You will configure cluster policies, manage entitlements, monitor performance, and contribute to CI/CD and IaC practices to ensure secure and scalable data pipelines.