Azure Databricks Engineer

Estuate, Inc.; Glendale (AZ); On-site; USD 140,000 - 180,000; Full time

Job summary

Estuate, Inc. is seeking a Senior Azure Databricks Engineer in Glendale, AZ for a 12-month engagement. The ideal candidate will boast 8–10 years in designing and implementing enterprise data platforms, with at least 5 years hands-on Azure Databricks in enterprise environments.

You will architect Lakehouse solutions, manage Unity Catalog, and ensure robust security, data governance, and CI/CD pipelines. Responsibilities include building data pipelines with PySpark/Spark SQL, implementing data

Qualifications

• 8–10 years designing and implementing enterprise data platforms.
• 5+ years hands-on Azure Databricks in enterprise environments.
• Experience with modern Lakehouse solutions using Azure Databricks and Delta Lake.
• Administering Azure Databricks workspaces, clusters, Unity Catalog and security.
• Developing data engineering solutions with PySpark/Spark SQL.
• Implementing Unity Catalog, metadata management, data lineage, access controls.
• Enterprise security architectures: RBAC, Managed Identities, Private Endpoints, Key Vault.
• Data quality frameworks, monitoring, observability, dashboards.

Skills

Azure Databricks

PySpark

Spark SQL

Data governance

RBAC & security

CI/CD

Python

SQL

Data modelling

Tools

Azure Databricks

Azure DevOps

Unity Catalog

Job description

Position: Senior Azure Databricks Engineer

Working Location: Glendale AZ

Duration : 12 Months

Experience: 8 to10 years

Requirements

• At least 8-10 years of experience in designing and implementing enterprise data platforms;
• At least 5 years of hands-on experience hands-on Azure Databricks experience in enterprise environments;
• Experience with architecting modern Lakehouse solutions using Azure Databricks and Delta Lake;
• Experience in administering Azure Databricks workspaces, clusters, Unity Catalog, compute policies, job orchestration, and workspace security;
• Experience in developing data engineering solutions using Apache Spark (PySpark and/or Spark SQL);
• Experience in implementing enterprise data governance utilizing Unity Catalog, metadata management, data lineage, and fine-grained access controls;
• Experience designing and working with enterprise security architectures including RBAC, Managed Identities, Private Endpoints, Key Vault integration, encryption, and network security;
• Experience implementing enterprise data quality frameworks, monitoring, observability, and operational dashboards;
• Strong understanding of data modelling techniques including dimensional modelling, Data Vault, medallion architecture, and enterprise data warehouse concepts;
• Strong knowledge of Azure Data Lake Storage Gen2, Azure Key Vault, Azure Monitor, Microsoft Entra ID (Azure AD), and Azure networking;
• Strong SQL programming skills and proficiency in Python and PySpark development
• Strong experience implementing CI/CD pipelines utilizing Azure DevOps, automated testing, release management, and deployment automation