Senior Data Engineer - Databricks & PySpark Pipelines

Robotics Technologies LLC; Madison (WI); On-site; USD 90,000 - 130,000; Full time

Job summary

Robotics Technologies LLC in Madison, WI seeks an experienced Data Engineer to design and maintain scalable data pipelines using Databricks and PySpark. The role focuses on ETL/ELT workflows with Python and SQL, leveraging Apache Spark, Delta Lake, and Unity Catalog to ensure data quality and accuracy.

You will integrate data from multiple sources, optimize processing, and implement validation processes across AWS/Azure/GCP cloud environments. The position is full-time and based in Madison, WI.

Qualifications

• Experience with Databricks, PySpark, Python and SQL.
• Proficient with cloud platforms (AWS/Azure/GCP).
• Experience building scalable data pipelines and ETL/ELT.

Responsibilities

• Develop data pipelines using Databricks and PySpark.
• Build ETL/ELT workflows using Python and SQL.
• Work with Apache Spark, Delta Lake, and Unity Catalog.
• Integrate data from multiple sources.
• Optimize data processing and SQL performance.
• Implement data quality and validation processes.

Skills

Data pipelines

ETL/ELT design

Data quality

Performance optimization

Tools

Databricks

PySpark

Python

SQL

Apache Spark

Delta Lake

Unity Catalog

AWS

Azure

GCP

Job description

Robotics Technologies LLC in Madison, WI seeks an experienced Data Engineer to design and maintain scalable data pipelines using Databricks and PySpark. The role focuses on ETL/ELT workflows with Python and SQL, leveraging Apache Spark, Delta Lake, and Unity Catalog to ensure data quality and accuracy.

You will integrate data from multiple sources, optimize processing, and implement validation processes across AWS/Azure/GCP cloud environments. The position is full-time and based in Madison, WI.