Senior Databricks Engineer for Enterprise Data Pipelines

Robotic Technologies Automation; Pennsylvania; On-site; USD 95,000 - 128,000; Full time

Job summary

Robotic Technologies Automation in Malvern, PA seeks a Senior Databricks Developer with 7+ years of data engineering to design and enhance enterprise data pipelines. The role focuses on Databricks, AWS, PySpark, Python, and SQL to build scalable, cloud-native solutions and production-ready data platforms.

The ideal candidate will implement Delta Lake and Medallion architectures, optimize batch and streaming jobs, and work with cross-functional teams to deliver high-quality data solutions.

Qualifications

• 7+ years of data engineering across enterprise data platforms.
• 5+ years hands-on Databricks development in production environments.
• Strong Python, PySpark, and SQL for pipeline development.
• Extensive experience building pipelines within AWS.
• Deep knowledge of Delta Lake, Lakehouse, and Medallion Architecture (Bronze, Silver, Gold).
• Experience with AWS services such as S3, Glue, Lambda, EMR, Redshift, Kinesis, IAM, and CloudWatch.
• Solid understanding of Spark architecture, distributed processing, and performance tuning.
• Experience building scalable batch and streaming data pipelines.
• Understanding of data modeling and modern data warehousing concepts.

Responsibilities

• Design, develop, and maintain large-scale ETL/ELT pipelines using Databricks, PySpark, Python, and SQL.
• Build scalable data ingestion, transformation, and orchestration frameworks in AWS.
• Develop and support Delta Lake implementations and Medallion Architecture pipelines.
• Create and optimize batch and streaming pipelines for large data volumes.
• Utilize AWS services (S3, Glue, Lambda, EMR, IAM, Kinesis, Redshift) to support data workflows.
• Optimize Spark jobs, Delta tables, and Databricks workloads for performance and cost.
• Collaborate with data engineers, architects, analytics teams, and stakeholders.
• Implement data quality controls, monitoring, and operational support.
• Troubleshoot production issues across Databricks and AWS environments.
• Contribute to engineering best practices and platform modernization.

Skills

Databricks

Python

PySpark

SQL

AWS

Delta Lake

Medallion Architecture

ETL/ELT pipelines

Data engineering

Job description

Robotic Technologies Automation in Malvern, PA seeks a Senior Databricks Developer with 7+ years of data engineering to design and enhance enterprise data pipelines. The role focuses on Databricks, AWS, PySpark, Python, and SQL to build scalable, cloud-native solutions and production-ready data platforms.

The ideal candidate will implement Delta Lake and Medallion architectures, optimize batch and streaming jobs, and work with cross-functional teams to deliver high-quality data solutions.