Databricks Data Engineer: Spark & Delta Lake Pipelines

Ethereum Technologies; Madison (WI); On-site; USD 110,000 - 160,000; Full time

Job summary

Ethereum Technologies in Madison, WI is seeking an experienced Data Engineer to design and maintain scalable data pipelines and ETL/ELT solutions using Databricks, PySpark, Python, and SQL.

You will work with Apache Spark, Delta Lake, and Unity Catalog to integrate data from multiple sources and optimize processing performance across cloud platforms (AWS/Azure/GCP). The role emphasizes data quality and validation in a collaborative environment.

Qualifications

• Experience building scalable data pipelines and ETL/ELT workflows.
• Strong SQL optimization and data modeling skills.
• Hands-on with Spark-based processing and cloud data platforms.

Responsibilities

• Develop data pipelines using Databricks and PySpark.
• Build ETL/ELT workflows using Python and SQL.
• Work with Apache Spark, Delta Lake, and Unity Catalog.
• Integrate data from multiple sources.
• Optimize data processing and SQL performance.
• Implement data quality and validation processes.
• Work with AWS, Azure, or GCP cloud platforms.

Skills

PySpark

Python

SQL

Databricks

Cloud platforms

Tools

Apache Spark

Delta Lake

Unity Catalog

Job description

Ethereum Technologies in Madison, WI is seeking an experienced Data Engineer to design and maintain scalable data pipelines and ETL/ELT solutions using Databricks, PySpark, Python, and SQL.

You will work with Apache Spark, Delta Lake, and Unity Catalog to integrate data from multiple sources and optimize processing performance across cloud platforms (AWS/Azure/GCP). The role emphasizes data quality and validation in a collaborative environment.