Lead Data Platform Engineer - Azure Databricks Lakehouse

Cactus Wellhead; Houston (TX); On-site; USD 150,000 - 210,000; Full time

Job summary

Cactus Wellhead in Houston is seeking a Data Platform Lead to design and govern the company’s Azure Databricks Lakehouse. You will own architecture, implement Bronze/Silver/Gold layers, Unity Catalog, and secure access, while ensuring scalable pipelines from ERP, PostgreSQL, SaaS, and APIs.

This role balances hands-on platform work with engineering leadership, setting standards, mentoring engineers, reviewing vendor work, and guiding production readiness, security, cost controls, and data

Qualifications

• 7+ years of experience in data engineering, data architecture, cloud data platforms, or enterprise data integration.
• Hands-on experience designing and supporting production data pipelines in cloud environments.
• Experience with modern Lakehouse architecture, preferably Azure Databricks and Delta Lake.
• Experience working with enterprise source systems such as ERP, CRM, SaaS applications, PostgreSQL, SQL Server, Oracle, or similar relational databases.
• Experience reviewing vendors or contractor technical deliverables and enforcing engineering standards.

Responsibilities

• Own the technical architecture for the Azure Databricks Lakehouse, including workspace structure, catalogs, schemas, compute patterns, storage strategy, and environment separation.
• Define and maintain bronze, silver, and gold layer standards, including naming conventions, table ownership, audit columns, refresh patterns, and production readiness criteria.
• Implement and govern Unity Catalog standards for access control, lineage, data classification, catalog/schema organization, and least‑privilege access.
• Partner with cybersecurity, infrastructure, and DevOps teams to align Databricks with enterprise identity, networking, secrets management, monitoring, and compliance expectations.
• Establish cost controls, cluster policies, job standards, and usage monitoring to ensure the platform is reliable, scalable, and cost‑effective.
• Design, build, and oversee production‑grade data pipelines using Databricks, Spark, Python/PySpark, SQL, Delta Lake, and approved orchestration patterns.
• Lead ingestion from ERP systems, product PostgreSQL databases, SaaS platforms, APIs, files, and other enterprise data sources into the Lakehouse.
• Define engineering patterns for full loads, incremental loads, CDC where applicable, reprocessing, error handling, logging, reconciliation, and pipeline recovery.
• Ensure every production pipeline includes source‑to‑target mapping, ownership, data quality rules, monitoring, alerting, and operational handover documentation.
• Review vendor and contractor deliverables for technical quality, maintainability, security, performance, and production readiness.

Skills

Azure Databricks

Apache Spark

Python PySpark

SQL

Delta Lake

Lakehouse

Unity Catalog

RBAC

Data Governance

CI/CD

Secrets Management

Cloud Storage

Data Modeling

Azure

Power BI

Tools

Azure DevOps

GitHub

Job description

Cactus Wellhead in Houston is seeking a Data Platform Lead to design and govern the company’s Azure Databricks Lakehouse. You will own architecture, implement Bronze/Silver/Gold layers, Unity Catalog, and secure access, while ensuring scalable pipelines from ERP, PostgreSQL, SaaS, and APIs.

This role balances hands-on platform work with engineering leadership, setting standards, mentoring engineers, reviewing vendor work, and guiding production readiness, security, cost controls, and data