Remote Data Platform Engineer (Databricks + Azure)

Greystar; United States; Remote; USD 125,000 - 150,000; Full time

Benefits offered by this job

Medical benefits

Dental benefits

Paid time off

401(k) match

Parental leave

Sabbatical after 10 years

Job summary

Greystar is seeking a Platform Engineer to join the Data team. This senior, technically focused role oversees Databricks workspace administration and Azure data services, with emphasis on governance, cost optimization, and robust DataOps capabilities.

You will manage CI/CD pipelines, Unity Catalog governance, and environment parity across Dev, Staging, and Production, while leveraging AI-driven observability and self-healing concepts.

Qualifications

• 5+ years of platform engineering, infrastructure engineering, or data operations in production environments.
• Expert-level Databricks administration: Unity Catalog, cluster policies, workspace management, compute configuration, Delta Lake internals.
• Strong command of the Azure data services ecosystem: ADLS Gen2, ADF, Azure Monitor, Log Analytics, Key Vault, Cost Management, Event Hub.
• Identity and access management: Azure Active Directory, service principals, RBAC, Unity Catalog governance policies.
• Hands-on experience with ADF pipeline design and orchestration at scale.
• Spark performance tuning: partitioning, Z-ordering, compute spend optimization, bottleneck profiling.
• Proven CI/CD experience using GitHub Enterprise: environment promotion, rollback, and release management for data pipelines.
• Python and/or SQL for automation scripts, compliance validation, and platform tooling.

Responsibilities

• Own Databricks workspace administration: upgrades, cluster policies, compute configurations, and Delta table lifecycle management.
• Administer Unity Catalog: access controls, service principal provisioning, governance policies, and data lineage.
• Tune Spark jobs for performance, reliability, and cost—profiling bottlenecks and optimizing partitioning.
• Architect and enforce DMP platform standards: naming conventions, schema evolution policies, SLA tiers, and medallion contracts.
• Leverage AI tooling to build DataOps capabilities including intelligent pipeline monitoring and anomaly detection.
• Operate the full Azure data services stack: ADLS Gen2, ADF, Azure Monitor, Key Vault, Log Analytics.
• Drive cost governance through Azure Cost Management, Databricks DBU optimization, and storage lifecycle policies.
• CI/CD for DMP data workflows: GitHub Enterprise workflows, environment-specific configuration, and release gating.
• Environment parity: IaC scripts and templates for reproducible provisioning across Dev/Staging/Prod.

Skills

Databricks admin

Azure data services

Identity management

GitHub Enterprise

Terraform IaC

Python/SQL

CI/CD pipelines

Azure DevOps

Tools

GitHub Enterprise

Terraform

Azure DevOps

Databricks

SQL

Job description

Greystar is seeking a Platform Engineer to join the Data team. This senior, technically focused role oversees Databricks workspace administration and Azure data services, with emphasis on governance, cost optimization, and robust DataOps capabilities.

You will manage CI/CD pipelines, Unity Catalog governance, and environment parity across Dev, Staging, and Production, while leveraging AI-driven observability and self-healing concepts.