Azure Databricks Platform Administrator at Smart IT Frame LLC Somerville, NJ

Healing Strides Counseling; Somerville (NJ); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Healing Strides Counseling seeks an experienced Azure Databricks Platform Administrator in New Jersey. Hybrid work arrangement with responsibilities spanning governance, workspace administration, IAM, networking, security, monitoring, automation, cost optimization, and production support.

The role collaborates with data engineering, analytics, security, cloud, and application teams to design, operate and optimize enterprise-scale Databricks environments on Azure.

Qualifications

• Bachelor's degree in CS or IT or related field.
• 5+ years in cloud data platform, DevOps or production support.
• Hands-on with Azure Databricks, Unity Catalog, Delta Lake, and Azure services.
• Experience with Terraform, ARM/Bicep, CI/CD pipelines and automation.

Responsibilities

• Administer Azure Databricks accounts, workspaces, clusters, policies, and access controls across environments.
• Provision and configure resources using IaC and automated pipelines.
• Implement IAM using Entra ID, SSO, service principals, managed identities, and RBAC.

Skills

Azure IAM

Networking security

ITIL processes

Cost optimization

Governance

Education

Bachelor's degree in CS/IT or related field

Tools

Terraform

Bicep

ARM templates

Azure CLI

Databricks CLI

PowerShell

Python

Shell scripting

Git

Azure DevOps

Job description

Position: Azure Databricks Platform Administrator

Location - Raritan, NJ (Hybrid)

Duration: Contract/Fulltime

Job Summary

We are seeking an experienced Azure Databricks Platform Administrator to design operate secure and optimize enterprise scale Databricks environments on Microsoft Azure The role is responsible for platform governance workspace administration identity and access management networking security monitoring automation cost optimization and production support The ideal candidate combines strong Azure infrastructure knowledge with hands on Databricks administration and works effectively with data engineering analytics security cloud and application teams

Key Responsibilities

• Administer Azure Databricks accounts workspaces clusters cluster policies pools SQL warehouses jobs notebooks repositories libraries and access controls across development test and production environments
• Manage workspace provisioning and configuration using Infrastructure as Code and automated deployment pipelines
• Implement identity and access management through Microsoft Entra ID single sign on service principals managed identities groups role-based access control and SCIM provisioning
• Administer Unity Catalog including metastores catalogs schemas tables volumes storage credential's external locations privileges and data access governance
• Configure secure Azure networking including virtual network injection private endpoints Private Link network security groups route tables DNS firewalls and controlled egress
• Integrate Databricks with Azure Data Lake Storage Key Vault Azure DevOps or GitHub monitoring platforms and other approved enterprise services
• Establish cluster standards runtime policies library controls tagging autoscaling auto termination and workload isolation
• Monitor platform availability performance capacity audit logs job health and security events build dashboards and actionable s
• Lead incident problem change release and service request management for the platform meeting agreed service levels and operational KPIs
• Troubleshoot complex workspace compute authentication authorization connectivity storage Spark and integration issues
• Optimize platform cost through usage analysis rightsizing policy enforcement commitment utilization charge back show back and removal of idle resources
• Plan and execute Databricks Runtime upgrades platform changes vulnerability remediation business continuity testing and disaster recovery activities
• Define and maintain platform standards runbooks architecture diagrams operating procedures support documentation and knowledge articles
• Partner with engineering architecture security compliance FinOps and vendor teams to onboard workloads and drive continuous improvement
• Mentor junior administrators and provide technical leadership for platform operations and governance

Required Qualifications and Experience

• 5 years of experience in cloud data platform infrastructure DevOps or production support roles including substantial hands‑on administration of Azure Databricks
• Strong understanding of Azure Databricks architecture control plane and data plane concepts workspaces compute jobs SQL warehouses Delta Lake and Unity Catalog
• Handson experience with core Azure services such as Microsoft Entra ID Azure Data Lake Storage Gen2 Key Vault Virtual Network Private Link Azure Monitor Log Analytics and Azure DevOps
• Strong knowledge of cloud security leastprivilege access secrets management encryption network isolation auditability and enterprise governance
• Experience automating Azure and Databricks administration using Terraform Bicep or ARM templates Databricks Asset Bundles Azure CLI Databricks CLI REST APIs PowerShell Python or shell scripting
• Experience building and supporting CICD pipelines for infrastructure and platform configuration
• Working knowledge of Apache Spark Python SQL Delta Lake and common dataengineering workload patterns
• Strong troubleshooting skills across identity networking compute storage security and distributed dataprocessing layers
• Experience operating production platforms using ITILaligned incident problem change release capacity and servicelevel management practices
• Bachelor's degree in computer science information technology engineering or a related discipline or equivalent practical experience

Technical Competencies

• Databricks Account Console workspace administration compute cluster policies jobs SQL warehouses Unity Catalog Delta Lake audit logs system tables authentication and REST APIs
• Azure Entra ID ADLS Gen2 Key Vault Virtual Network Private Link Azure Monitor Log Analytics Azure Policy and Azure DevOps
• Automation and DevOps Terraform Bicep or ARM Git CICD Databricks Asset Bundles PowerShell Python shell scripting and com

Skills

Mandatory Skills : AZURE DATA LAKE, Azure SQL, Databricks