Databricks Lakehouse Architect - AWS + Delta & Governance

Machine Learning Technologies, LLC; Atlanta (GA); On-site; USD 140,000 - 180,000; Full time

Job summary

Machine Learning Technologies, LLC in Atlanta seeks a highly experienced Databricks Architect to design, implement, and optimize enterprise Databricks solutions on AWS. You will lead Lakehouse architectures, Delta Lake, Unity Catalog, and Spark-based data platforms to deliver secure, scalable analytics.

You'll build end-to-end data pipelines, orchestration with Databricks Workflows, and ensure high performance for batch and streaming workloads, integrating S3/IAM/Glue, and advocating best

Qualifications

• Extensive hands-on Databricks development and architecture experience.
• Strong Databricks Lakehouse, Delta Lake, and PySpark expertise.
• Experience architecting enterprise-scale Databricks solutions on AWS.
• Candidates lacking Databricks hands-on expertise in AWS will not be considered.

Responsibilities

• Design end-to-end enterprise Databricks Lakehouse architecture on AWS.
• Build and optimize scalable data pipelines with Databricks and Spark.
• Develop ETL/ELT workflows using PySpark and Spark SQL.
• Implement Delta Lake and Unity Catalog for governance.
• Optimize Spark jobs and cluster configurations.
• Implement Databricks Workflows for orchestration.
• Develop batch and streaming data processing solutions.
• Lead Databricks platform architecture and design.

Skills

Databricks on AWS

Apache Spark

PySpark

Spark SQL

Delta Lake

Unity Catalog

Databricks Workflows

ETL/ELT pipelines

Batch processing

Streaming processing

Medallion Architecture

Python

SQL

Tools

AWS Glue

Amazon S3

IAM

Job description

Machine Learning Technologies, LLC in Atlanta seeks a highly experienced Databricks Architect to design, implement, and optimize enterprise Databricks solutions on AWS. You will lead Lakehouse architectures, Delta Lake, Unity Catalog, and Spark-based data platforms to deliver secure, scalable analytics.

You'll build end-to-end data pipelines, orchestration with Databricks Workflows, and ensure high performance for batch and streaming workloads, integrating S3/IAM/Glue, and advocating best