Senior Databricks Data Engineer - AWS Lakehouse

Robotic Technologies Automation; Harrisburg (Dauphin County); On-site; USD 83,000 - 131,000; Full time

Job summary

Robotic Technologies Automation in Harrisburg, PA seeks a senior Databricks Data Engineer to design and implement a scalable Sales Data Platform on Databricks AWS. The role is hands-on and architecture-driven, focusing on data ingestion, transformation, modelling, and optimization using lakehouse patterns.

You will define end-to-end Lakehouse architecture for Sales data, establish data quality, governance, and CI/CD pipelines, and mentor engineers while collaborating with source system teams and

Qualifications

• 8+ years in Data Engineering / Data Architecture roles.
• 4+ years hands-on experience with Databricks.
• Strong expertise in PySpark & Spark SQL.
• Deep experience with Delta Lake.
• Strong knowledge of AWS cloud services (S3, IAM, Glue, CloudWatch).
• Strong understanding of: Fact & dimension modelling, Slowly Changing Dimensions (SCD Type 1 / 2).
• Experience handling high-volume, high-velocity datasets.

Responsibilities

• Lead end-to-end Lakehouse design on Databricks (AWS) for Sales data domains.
• Design medallion architecture (Bronze / Silver / Gold) and data modelling standards.
• Establish scalable ingestion patterns for batch and incremental loads.
• Drive performance, scalability, and cost optimization best practices.
• Build and guide PySpark-based data pipelines in Databricks; ensure ACID, idempotent pipelines.
• Implement CDC and incremental processing with watermarking and change detection.
• Provide technical leadership and mentorship to Databricks data engineers.

Skills

Databricks

PySpark

Spark SQL

Delta Lake

AWS (S3, IAM, Glue, CloudWatch)

Data Modeling

Lakehouse Architecture

Python

ETL Pipelines

Tools

Spark

S3

IAM

AWS Glue

Delta Lake

Job description

Robotic Technologies Automation in Harrisburg, PA seeks a senior Databricks Data Engineer to design and implement a scalable Sales Data Platform on Databricks AWS. The role is hands-on and architecture-driven, focusing on data ingestion, transformation, modelling, and optimization using lakehouse patterns.

You will define end-to-end Lakehouse architecture for Sales data, establish data quality, governance, and CI/CD pipelines, and mentor engineers while collaborating with source system teams and