Senior Data Architect - Databricks Lakehouse (Remote)

HMG AMERICA LLC; United States; Remote; USD 140,000 - 180,000; Full time

Job summary

HMG AMERICA LLC is seeking a Lead Data Engineer (Databricks) to architect and deliver enterprise-scale data platforms on AWS using Databricks. You will guide architecture, governance, and cost optimization while engaging client stakeholders as a trusted advisor.

The role requires hands-on Databricks expertise, Delta Live Tables, Unity Catalog, and a strong background in IT consulting and modern data lakehouse patterns.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Technology, or a related field.
• 8+ years in Data Engineering with at least 3+ years in a Lead/Architect capacity.
• Deep experience with Databricks, PySpark/Scala, Delta Lake, Unity Catalog, and Delta Live Tables (DLT).
• Hands-on experience with AWS Cloud Data Ecosystem (S3, Glue, Lambda, EMR, Redshift, IAM).
• Proficiency in Python, SQL, and Spark SQL performance tuning.
• Strong experience with Infrastructure-as-Code (Terraform) and CI/CD tools (GitHub Actions, Azure DevOps, Jenkins).
• Consulting & Soft Skills: Exceptional verbal and written communication; client-facing advisory experience.

Responsibilities

• Architect, design, and implement scalable data platforms on AWS using Databricks Lakehouse architecture.
• Establish Bronze-Silver-Gold patterns for real-time and batch pipelines.
• Implement data governance, access control, and lineage using Unity Catalog and IAM.
• Optimize cluster configurations, Delta Lake performance, and costs.
• Integrate AWS services with Databricks and manage multi-environment CI/CD pipelines.

Skills

Databricks

PySpark/Scala

Delta Lake

Unity Catalog

Delta Live Tables

AWS

Python

SQL

Spark SQL

Consulting

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

Terraform

GitHub Actions

Azure DevOps

Jenkins

Job description

HMG AMERICA LLC is seeking a Lead Data Engineer (Databricks) to architect and deliver enterprise-scale data platforms on AWS using Databricks. You will guide architecture, governance, and cost optimization while engaging client stakeholders as a trusted advisor.

The role requires hands-on Databricks expertise, Delta Live Tables, Unity Catalog, and a strong background in IT consulting and modern data lakehouse patterns.