Remote Senior Databricks Architect & Data Platform Lead

SDH Systems; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

SDH Systems is seeking a Lead Databricks Data Engineer / Databricks Architect to design and implement scalable Lakehouse architectures and enterprise data pipelines in a 100% remote role.

The candidate will lead technical projects, optimize Delta Lake, and deliver robust data governance using Unity Catalog across cloud platforms (AWS/Azure/GCP). Strong leadership and 10+ years in data engineering are required.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Technology, or related field.
• 10+ years of experience in Data Engineering / Big Data / Analytics.
• 4+ years of hands-on Databricks experience preferred.
• Strong experience designing enterprise-scale data platforms.
• Demonstrated experience leading technical projects and mentoring engineers.
• Strong communication and stakeholder-management skills.

Responsibilities

• Design and develop scalable data engineering solutions using Databricks and Lakehouse architecture.
• Build robust ETL/ELT pipelines using PySpark, Spark SQL, Python, and SQL.
• Design and implement Bronze, Silver, and Gold/Medallion architecture.
• Develop and optimize Delta Lake tables with MERGE, schema evolution, and CDC.
• Build batch and real-time pipelines using Structured Streaming and Lakeflow.
• Develop and manage Databricks Jobs/Workflows for orchestration and monitoring.
• Implement Unity Catalog governance, access control, lineage, auditing.

Skills

Databricks

Delta Lake

Unity Catalog

Lakehouse

PySpark

SQL

GCP/AWS/Azure

ETL/ELT

CI/CD

Terraform

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks Lakehouse Platform

Delta Live Tables

Auto Loader

Lakeflow

Photon

Terraform

GitHub/Jenkins

Job description

SDH Systems is seeking a Lead Databricks Data Engineer / Databricks Architect to design and implement scalable Lakehouse architectures and enterprise data pipelines in a 100% remote role.

The candidate will lead technical projects, optimize Delta Lake, and deliver robust data governance using Unity Catalog across cloud platforms (AWS/Azure/GCP). Strong leadership and 10+ years in data engineering are required.