BI Admin Databricks

VSG Business Solutions LLC; Charlotte (ME); On-site; USD 120,000 - 180,000; Full time

Job summary

VSG Business Solutions LLC is seeking an experienced Databricks Platform Administrator to manage the Databricks workspace across development, test, and production environments. You will handle user provisioning, RBAC, service principals, and governance controls while configuring policies, clusters, and compute resources.

You will monitor platform health, implement security controls, and support Unity Catalog, data governance, and workspace administration.

Qualifications

• Azure Cloud experience required.
• Strong IAM knowledge: SSO, SCIM, RBAC, service principals.
• Experience with Databricks cluster management, workspace administration, job orchestration, and governance controls.
• Knowledge of networking: VNet/VPC, private endpoints, firewalls, DNS, secure connectivity.
• Experience scripting/automation with Python, PowerShell, Bash, Terraform, or similar tools.
• Familiarity with monitoring, logging, alerting, and incident response processes.
• Strong troubleshooting, communication, and cross-functional collaboration skills.

Responsibilities

• Administer and support the Databricks workspace environment across development, test, and production.
• Manage user provisioning, RBAC, service principals, groups, and entitlements.
• Configure and maintain cluster policies, instance pools, job compute, and resources.
• Monitor platform health, performance, usage trends, and incidents.
• Implement security controls for secrets management, data access, encryption, private connectivity, and audit logging.
• Support Unity Catalog, data governance standards, and workspace administration.
• Develop and maintain automation for workspace setup, policy enforcement, monitoring, and admin tasks.
• Support CI/CD and IaC for Databricks deployments and config management.
• Troubleshoot platform, connectivity, identity federation, and job execution issues.
• Create and maintain runbooks, standards, and support procedures.
• Understand data engineering workflows with Spark, notebooks, Delta Lake, and scheduling.

Skills

Databricks Workspace Administration

Identity & Access Management

Databricks governance & Unity Catalog

Scripting & Automation

Networking concepts

Monitoring & Incident Response

Troubleshooting & Collaboration

Tools

Terraform

Python

PowerShell

Bash

Job description

Job Description

Key Responsibilities

• Administer and support the Databricks workspace environment across development, test, and production.
• Manage user provisioning, role based access controls, service principles, groups, and entitlements.
• Configure and maintain cluster policies, instance pools, job compute, and all purpose compute resources.
• Monitor platform health, job performance, usage trends, and operational incidents.
• Implement security controls for secrets management, data access, encryption, private connectivity, and audit logging.
• Support Unity Catalog, data governance standards, and workspace level administration.
• Develop and maintain automation for workspace setup, policy enforcement, monitoring, and administrative tasks.
• Support CI/CD and infrastructure as code practices for Databricks deployments and configuration management.
• Troubleshoot platform, connectivity, identity federation, and job execution issues.
• Create and maintain operational documentation, runbooks, standards, and support procedures.
• Understanding data engineering workflows using Spark, notebooks, Delta Lake, and job scheduling.

Required Qualifications

• Experience with Azure Cloud.
• Strong understanding of identity and access management, including SSO, SCIM, RBAC, and service principals.
• Experience with Databricks cluster management, workspace administration, job orchestration, and governance controls.
• Knowledge of networking concepts such as VNet/VPC, private endpoints, firewalls, DNS, and secure connectivity.
• Experience with scripting or automation using Python, PowerShell, Bash, Terraform, or similar tools.
• Familiarity with monitoring, logging, alerting, and incident response processes.
• Strong troubleshooting, communication, and cross functional collaboration skills.

Key Skills

• Databricks Workspace Administration
• Unity Catalog and Governance
• Cluster Policies and Performance Tuning
• Cloud Platform Integration
• Security and Access Management
• Automation and Infrastructure as Code
• Cost Optimization
• Troubleshooting and Operational Support
• Documentation and Cross Functional Collaboration
• Logic Apps, APIM, App Gateways