Senior Databricks Engineer — Cloud Data Platform Lead

Ethereum Technologies LLC; Pennsylvania; On-site; USD 112,000 - 128,000; Part time

Job summary

Ethereum Technologies in Malvern, PA seeks a Senior Databricks Developer with 7+ years of data engineering experience. You will design and optimize enterprise data pipelines using Databricks, PySpark, Python, and SQL on AWS, implementing Delta Lake and Medallion architectures across Bronze-Silver-Gold layers.

The role is highly hands-on, cloud-native, and focused on scalable batch and streaming processing. Expect collaboration with architects and analytics teams to deliver robust data platforms.

Qualifications

• 7+ years of Data Engineering experience building enterprise-scale data platforms and pipelines.
• 5+ years of hands-on Databricks development experience in production environments.
• Strong expertise in Python, PySpark, and SQL with a focus on pipeline development.
• Extensive experience developing and supporting data pipelines within AWS.
• Strong knowledge of Delta Lake, Lakehouse Architecture, and Medallion Architecture (Bronze, Silver, Gold).
• Hands-on experience with AWS services such as S3, Glue, Lambda, EMR, Redshift, Kinesis, IAM, and CloudWatch.

Responsibilities

• Design, develop, and maintain large-scale ETL/ELT pipelines using Databricks, PySpark, Python, and SQL.
• Build scalable and highly reliable data ingestion, transformation, and orchestration frameworks within AWS.
• Develop and support Delta Lake implementations and Medallion Architecture (Bronze, Silver, Gold) data pipelines.
• Create and optimize batch and streaming pipelines that process large volumes of structured and unstructured data.
• Leverage AWS services including S3, Glue, Lambda, EMR, IAM, EventBridge, CloudWatch, Kinesis, and Redshift to support end-to-end data workflows.
• Optimize Spark jobs, Delta tables, and Databricks workloads for performance, scalability, and cost efficiency.
• Partner with Data Engineers, Architects, Analytics teams, and business stakeholders to deliver high-quality data solutions.
• Implement data quality controls, monitoring, alerting, and operational support processes.
• Troubleshoot and resolve complex production issues across Databricks and AWS environments.
• Contribute to engineering best practices, code reviews, automation, and platform modernization initiatives.

Skills

Databricks

PySpark

Python

SQL

AWS

Data Engineering

Delta Lake

Medallion Architecture

Tools

Databricks

PySpark

Python

SQL

AWS

Job description

Ethereum Technologies in Malvern, PA seeks a Senior Databricks Developer with 7+ years of data engineering experience. You will design and optimize enterprise data pipelines using Databricks, PySpark, Python, and SQL on AWS, implementing Delta Lake and Medallion architectures across Bronze-Silver-Gold layers.

The role is highly hands-on, cloud-native, and focused on scalable batch and streaming processing. Expect collaboration with architects and analytics teams to deliver robust data platforms.