Azure Databricks Admin

E-Solutions INC; Plano (TX); On-site; USD 90,000 - 130,000; Full time

Job summary

E-Solutions INC is seeking an Azure Databricks Administrator to lead the platform across Development, QA, Production, and DR environments. The candidate will manage Unity Catalog governance, RBAC, and data access policies while implementing automation and security controls.

Responsibilities include workspace administration, upgrades, migrations, and incident response, with strong emphasis on cost control, monitoring, and enterprise governance in a high-availability cloud environment.

Qualifications

• Extensive architecture and admin experience with Azure Databricks.
• Experience with platform automation, DevOps, and CI/CD pipelines.
• Knowledge of cloud security, networking, and identity management.
• Experience managing Unity Catalog and data access control.
• Experience with RBAC, SCIM provisioning, and user lifecycle management.

Responsibilities

• Administer Azure Databricks workspaces across multiple environments.
• Implement capacity planning, DR, and HA strategies.
• Configure and manage Unity Catalog permissions and governance.
• Integrate Databricks with Azure AD and manage security tooling.
• Establish monitoring, alerting, and runbooks; support upgrades and migrations.

Skills

Azure Databricks administration

Terraform IaC

CI/CD pipelines

Unity Catalog governance

RBAC and identity mgmt

Cloud security & networking

Kubernetes & VMs

Monitoring & observability

Data security & access control

Disaster recovery & HA

Education

Cloud/Databricks certifications

Tools

Azure AD

Key Vault

Terraform

Unity Catalog

Job description

Position: Azure Databricks Admin

Type/Duration: Full time//Permanent

Location: Plano, TX/Richardson, TX (5 Days onsite in a week)

Required Skill and Experience

• Architecture and administration experience with Microsoft Azure Databricks
• Experience with platform automation, DevOps, and CI/CD pipelines
• Knowledge of cloud security, networking, and identity management
• Establish platform monitoring, observability, and automated alerting frameworks for Azure Databricks platform components
• Platform automation through Infrastructure-as-Code using Terraform, DevOps, and CI/CD pipelines.
• Implement capacity planning, disaster recovery, and high availability strategies.
• Expertise in managing Unity Catalog, and data access control mechanisms.
• Experience of managing Databricks cluster policies, instance pools, autoscaling, auto-termination, and compute governance
• Support platform upgrades, runtime migrations, and feature enablement.
• Administer Azure Databricks workspaces across Development, QA, Production & DR environments.
• Integrate Azure Databricks with Microsoft Entra ID (Azure AD).
• Manage RBAC, SCIM provisioning, user lifecycle management, and group synchronization
• Implement and maintain Unity Catalog permissions and governance policies.
• Configure Key Vault-backed secret scopes and credential management.
• Support secure networking using:
  • Private Endpoints
  • VNET Injection
  • NSGs
  • Firewall Rules
  • Managed Identities
• Implement enterprise governance standards using Unity Catalog.
• Maintain audit logging and compliance reporting.
• Troubleshoot platform, cluster, notebook, and job execution issues.
• Establish operational runbooks, incident management procedures, and SLA monitoring.
• Implement cluster policies to control costs and prevent resource overprovisioning.

Preferred Skill and Experience

• Manage platform operations, incident management, and change governance
• Design scalable, resilient, and secure platform solutions aligned with enterprise architecture standards.
• Establish best practices for platform architecture, governance, and security.
• Certifications in cloud architecture (Azure, Databricks) preferred
• ITIL knowledge for service management and operations governance
• Deliver platform health metrics, risk assessments, and operational reports.
• Ensure adherence to SLA, compliance, and operational standards
• Experience of supporting migration of legacy systems (like HMS) to Databricks on cloud platforms like Azure, AWS, or GCP.
• Skillset of integrating guidelines to configure and integrate Databricks with other existing security tools relevant to data access control.
• Experience in Azure Databricks, real time data ingestion, data warehouses, serverless ETL, No SQL databases, DevOps, Kubernetes, virtual machines, web/function apps, monitoring, and security tools.
• Optimize cluster utilization and DBU consumption.
• Utilize autoscaling, instance pools, spot instances, and workload optimization techniques.
• Produce usage, cost, and governance dashboards for stakeholders.
• Provide guidance on Databricks best practices and platform standards.
• Support onboarding of new projects and workloads onto the platform.
• Managed Kubernetes cluster and workload including network architecture