Databricks Lakehouse Architect AWS Data Leader

OMG Technology; Village of Tarrytown (NY); Hybrid; USD 160,000 - 210,000; Full time

Job summary

OMG Technology in New York is seeking a Databricks Architect to lead enterprise-scale Lakehouse architecture and implementation on AWS. The role focuses on designing scalable data solutions, modernizing existing platforms, establishing governance standards, and guiding customer discussions.

The candidate will drive migration of legacy ETL workloads to Databricks, define target-state architectures, and develop reusable patterns.

Qualifications

• 15+ years of data & analytics experience.
• 4+ years hands-on Databricks experience.
• Strong experience with Databricks, Delta Lake, Unity Catalog, DLT, and Auto Loader.
• Proficiency in PySpark, Spark SQL, Python, and Structured Streaming.
• AWS experience with S3, IAM, Glue, Redshift, EMR, and Lambda.
• Experience with Airflow, CDC, and enterprise data migration.
• Knowledge of data governance, security, and observability.
• Strong customer-facing, solution architecture, and technical leadership skills.

Responsibilities

• Lead enterprise-scale Databricks Lakehouse architecture and implementation on AWS.
• Drive migration of EMR, NiFi, Redshift, and legacy ETL workloads to Databricks.
• Evaluate newer Databricks capabilities, including Serverless, Lakehouse Federation, Genie, Databricks Apps, Lakebase, and AI/ML.
• Architect scalable batch, streaming, CDC, and real-time data solutions.
• Design reusable frameworks using Delta Lake, PySpark, Auto Loader, DLT, and Databricks Workflows.
• Define governance, security, lineage, and data-quality standards using Unity Catalog.
• Establish best practices for performance, scalability, and cost optimization.
• Lead customer workshops, architecture reviews, and solution assessments.
• Define target-state architecture, migration roadmaps, and reusable patterns.

Skills

Databricks

PySpark

Spark SQL

Python

Structured Streaming

AWS

Data governance

Leadership

Solution architecture

Customer-facing

Tools

Delta Lake

Unity Catalog

DLT

Auto Loader

Airflow

Databricks

AWS S3

Redshift

EMR

Glue

Job description

OMG Technology in New York is seeking a Databricks Architect to lead enterprise-scale Lakehouse architecture and implementation on AWS. The role focuses on designing scalable data solutions, modernizing existing platforms, establishing governance standards, and guiding customer discussions.

The candidate will drive migration of legacy ETL workloads to Databricks, define target-state architectures, and develop reusable patterns.