Databricks Data Engineer - Lakehouse Architect (AWS)

Quantum Technologies. LLC; Harrisburg (Dauphin County); On-site; USD 83,000 - 124,000; Part time

Job summary

Quantum Technologies. LLC is seeking a Databricks Data Engineer to lead the design and implementation of a scalable Sales Data Platform on Databricks running on AWS, based in Harrisburg, PA.

This contract role (C2C/1099) spans 10 months with a bill rate of $75/hour. You will build PySpark pipelines, implement Bronze/Silver/Gold Lakehouse architecture, ensure data quality and governance, and drive performance, cost optimization, and CI/CD for notebooks and jobs.

Qualifications

• 8+ years in Data Engineering / Data Architecture roles.
• 4+ years hands-on experience with Databricks.
• Strong expertise in PySpark & Spark SQL.
• Deep experience with Delta Lake.
• Strong knowledge of AWS cloud services (S3, IAM, Glue, CloudWatch).
• Strong understanding of Fact & dimension modelling and SCD Type 1/2.
• Experience handling high-volume, high-velocity datasets.

Responsibilities

• Lead design and implementation of scalable Sales Data Platform on Databricks (AWS).
• Build PySpark-based data pipelines and medallion architecture (Bronze/Silver/Gold).
• Define end-to-end Lakehouse architecture and data modelling standards for Sales data domains.
• Implement CDC, incremental processing, and robust error handling.
• Develop CI/CD pipelines for Databricks notebooks and jobs and mentor engineers.

Skills

Data engineering

Data architecture

PySpark & Spark SQL

Delta Lake

AWS cloud services

Data modelling

SCD Type 1/2

Large-scale data processing

Tools

Databricks

AWS S3

AWS IAM

AWS Glue

CloudWatch

Delta Lake

Job description

Quantum Technologies. LLC is seeking a Databricks Data Engineer to lead the design and implementation of a scalable Sales Data Platform on Databricks running on AWS, based in Harrisburg, PA.

This contract role (C2C/1099) spans 10 months with a bill rate of $75/hour. You will build PySpark pipelines, implement Bronze/Silver/Gold Lakehouse architecture, ensure data quality and governance, and drive performance, cost optimization, and CI/CD for notebooks and jobs.