Databricks Administrator

Insight Global; Charlotte (NC); On-site; USD 140,000 - 190,000; Full time

Job summary

Insight Global is seeking a Databricks Platform Engineer to own provisioning, configuration, and ongoing management of Databricks environments within Azure, ensuring secure, scalable, and standardized platform operations.

You will establish workspace standards, manage cluster policies and networking, onboard new data sources, integrate with Azure services, implement RBAC and governance, and automate infrastructure with Terraform and CI/CD while collaborating with data engineering teams to enable

Qualifications

• Strong hands-on experience administering Azure Databricks in enterprise environments.
• Provisioning, configuring, and managing Databricks workspaces.
• Deep knowledge of Azure services and cloud-based data platforms.
• Experience managing Databricks workspaces, clusters, and SQL Warehouses.
• Security best practices including RBAC and governance.

Responsibilities

• Provision and manage Databricks environments in Azure.
• Establish workspace standards, policies, and networking integrations.
• Implement RBAC, access controls and governance controls.
• Automate infrastructure with Terraform and CI/CD pipelines.
• Collaborate with data engineering teams to enable data pipelines.

Skills

Azure Databricks administration

Terraform IaC

Azure cloud services expertise

Databricks workspace/cluster managing

RBAC & security best practices

Data governance (Unity Catalog)

CI/CD for data platforms

Data ingestion & integration

Cost optimization & performance tuning

Collaboration with cross-functional

Tools

Databricks CLI

Azure DevOps

GitHub Copilot

Job description

This Databricks Platform Engineer will own the provisioning, configuration, and ongoing management of Databricks environments within Azure, ensuring secure, scalable, and standardized platform operations. Responsibilities include establishing workspace standards, managing cluster policies and networking integrations, onboarding new data sources, integrating Databricks with Azure services such as ADLS, Cosmos DB, and PostgreSQL, implementing RBAC and governance controls, automating infrastructure through Terraform and CI/CD pipelines, leveraging AI-powered development tools to improve efficiency, monitoring platform performance and cost optimization, and partnering with data engineering teams to enable data pipelines while translating business and technical requirements into scalable platform solutions.

Required Skills & Experience

• Strong hands-on experience administering Azure Databricks in enterprise environments
• Proven experience provisioning, configuring, and managing Databricks workspaces
• Deep knowledge of Azure services and cloud-based data platforms
• Experience managing Databricks workspaces, clusters, and SQL Warehouses
• Strong understanding of compute optimization, performance tuning, and cost management
• Hands-on experience with Terraform for infrastructure as code (IaC)
• Experience provisioning Azure and Databricks infrastructure through automation
• Expertise building and maintaining automated environment setup and configuration processes
• Experience implementing RBAC, access controls, and security best practices
• Experience establishing data governance frameworks, preferably Unity Catalog
• Experience onboarding new data sources and enabling scalable ingestion patterns
• Ability to partner effectively with data engineering, application, and cross-functional teams
• Strong problem-solving skills and the ability to translate business requirements into platform solutions

Nice to Have Skills & Experience

• Experience using AI coding tools such as GitHub Copilot, ChatGPT, or Claude for code generation and automation
• Databricks certifications, preferably Databricks Platform Administrator
• Experience with MLflow, Delta Live Tables (DLT), and Databricks Workflows
• Experience supporting machine learning teams and onboarding ML Engineers into Databricks workspaces
• Exposure to data, analytics, and AI platform enablement within Databricks environments