Remote Databricks Platform Engineer — Azure DataOps

greystar; United States; Remote; USD 125,000 - 150,000; Full time

Benefits offered by this job

401(k) with company match

Paid parental leave

Sabbatical after 10 years

Flexible work arrangements

Job summary

Greystar seeks a Databricks Platform Engineer to lead administration of Databricks workspace, Unity Catalog governance, and Azure infrastructure across the enterprise data stack. This senior, platform-focused role emphasizes reliability, cost controls, and platform standards within a Databricks-native medallion architecture on Azure.

You will own CI/CD pipelines for data workflows, manage IAM and secret governance, optimize Spark performance, and collaborate with cross-functional teams to

Qualifications

• 5+ years of platform/infra/data ops experience in production.
• Expert Databricks administration with Unity Catalog.
• Deep knowledge of Azure data services and cost governance.
• IAM with Azure AD, RBAC, Unity Catalog governance.
• ADF design and orchestration at scale.
• Spark performance tuning and cost optimization.
• CI/CD with GitHub Enterprise for data pipelines.
• Automation using Python and/or SQL.
• Knowledge of medallion/lakehouse patterns.
• IaC with Terraform for Databricks and Azure resources.
• Azure DevOps pipelines and YAML workflow.
• Networking, private endpoints, and DNS resolution.
• Azure Monitor, Log Analytics for observability.
• Delta Lake lifecycle management and VACUUM/OPTIMIZE

Responsibilities

• Own Databricks workspace administration, upgrades, and policies.
• Administer Unity Catalog access controls and provenance.
• Tune Spark jobs for performance, cost, and reliability.
• Enforce DMP platform standards and medallion contracts.
• Leverage AI features like Mosaic AI for dataops capabilities.
• Operate full Azure data services stack and monitor spend.
• Provide platform support across multiple systems and platforms.
• Own CI/CD for DMP data workflows from dev to prod.
• Manage IAM, secret rotation, and governance documentation.
• Maintain IaC scripts for reproducible environments.

Skills

Databricks admin

Unity Catalog

Azure data services

Azure AD RBAC

ADF orchestration

Spark tuning

GitHub CI/CD

Python/SQL

Delta Lake

IaC / Terraform

Azure DevOps

Networking

Log Analytics

Delta Lake lifecycle

Tools

Databricks

Azure

GitHub Enterprise

Job description

Greystar seeks a Databricks Platform Engineer to lead administration of Databricks workspace, Unity Catalog governance, and Azure infrastructure across the enterprise data stack. This senior, platform-focused role emphasizes reliability, cost controls, and platform standards within a Databricks-native medallion architecture on Azure.

You will own CI/CD pipelines for data workflows, manage IAM and secret governance, optimize Spark performance, and collaborate with cross-functional teams to