Databricks Lakehouse Architect – AWS Expert

Artifintech; Atlanta (GA); On-site; USD 150,000 - 230,000; Full time

Job summary

Artifintech is seeking a highly experienced Databricks Architect to design, build, and optimize enterprise-scale Databricks solutions on AWS. The role demands deep hands-on expertise across Lakehouse architecture, Delta Lake, Unity Catalog, Apache Spark, and Databricks Workflows.

You will lead end-to-end architecture, develop scalable data pipelines, and collaborate with data engineering and cloud teams to deliver high-performance, secure data platforms.

Qualifications

• Extensive hands-on Databricks development and architecture experience.
• Deep expertise in Databricks Lakehouse, Delta Lake, and PySpark.
• Experience designing enterprise-scale Databricks solutions on AWS.
• Strong knowledge of Spark performance tuning.
• Proven ability to lead architecture and design discussions.

Responsibilities

• Design and implement end-to-end enterprise Databricks Lakehouse architecture on AWS.
• Build and optimize scalable data pipelines using Databricks and Apache Spark.
• Develop complex ETL/ELT workflows using PySpark and Spark SQL.
• Implement and manage Delta Lake architecture, including Delta Tables and Delta Live Tables.
• Design and implement Unity Catalog for centralized data governance, access control, and metadata management.
• Optimize Spark jobs, cluster configurations, partitioning, caching, and query performance.
• Implement Databricks Workflows for job orchestration and automation.
• Develop scalable batch and streaming data processing solutions.
• Establish Medallion Architecture across Bronze, Silver, and Gold layers.
• Integrate Databricks with AWS services such as S3, IAM, Glue.
• Establish best practices for data quality and monitoring.
• Lead Databricks platform architecture and implementation.
• Collaborate with data engineering and enterprise architecture teams.

Skills

Databricks expertise

Python

SQL

ETL/ELT design

Data governance

Tools

Databricks on AWS

Apache Spark

PySpark

Delta Lake

Unity Catalog

Delta Live Tables

Lakeflow

Medallion Architecture

AWS S3/IAM/Glue

Job description

Artifintech is seeking a highly experienced Databricks Architect to design, build, and optimize enterprise-scale Databricks solutions on AWS. The role demands deep hands-on expertise across Lakehouse architecture, Delta Lake, Unity Catalog, Apache Spark, and Databricks Workflows.

You will lead end-to-end architecture, develop scalable data pipelines, and collaborate with data engineering and cloud teams to deliver high-performance, secure data platforms.