Databricks Data Engineer – Lakehouse Architect on AWS

Machine Learning Technologies, LLC; Harrisburg (Dauphin County); On-site; USD 63,000 - 89,000; Full time

Job summary

Machine Learning Technologies, LLC in Harrisburg, PA is seeking a Databricks Data Engineer to lead the design and implementation of a scalable Sales Data Platform on Databricks running on AWS. The role is hands-on and architecture-driven, focusing on data ingestion, transformation, modelling, and optimization using modern Lakehouse patterns.

The architect will define end-to-end Lakehouse architecture on Databricks (AWS) for Sales data domains and design medallion (Bronze/Silver/Gold) layers,

Qualifications

• Experience designing and implementing data platforms on Databricks.
• Strong skills in data ingestion, transformation, and modelling.
• Proficiency with AWS-based data lakehouse architectures.

Responsibilities

• Lead design and implementation of a scalable Sales Data Platform on Databricks (AWS).
• Build and optimize PySpark-based data pipelines with ACID transactions.
• Define end-to-end Lakehouse architecture and Bronze/Silver/Gold layers.
• Establish scalable ingestion patterns for batch and incremental loads.
• Ensure performance, scalability, and cost optimization of pipelines.

Skills

Databricks

PySpark

Data modeling

Ingestion patterns

AWS

Tools

SQL

Spark

Job description

Machine Learning Technologies, LLC in Harrisburg, PA is seeking a Databricks Data Engineer to lead the design and implementation of a scalable Sales Data Platform on Databricks running on AWS. The role is hands-on and architecture-driven, focusing on data ingestion, transformation, modelling, and optimization using modern Lakehouse patterns.

The architect will define end-to-end Lakehouse architecture on Databricks (AWS) for Sales data domains and design medallion (Bronze/Silver/Gold) layers,