Databricks/Python Data Engineer — Scalable ETL & Lakehouse Architect

OpenTalent; Town of Hyde Park (NY); Hybrid; USD 110,000 - 170,000; Part time

Job summary

Databricks is seeking a Databricks / Python Data Engineer to design, develop, and optimize enterprise-scale data pipelines and Lakehouse solutions using Python, Databricks, Apache Spark and PySpark. The role focuses on building scalable ETL/ELT processes and implementing medallion architectures for reliable data maturity.

The candidate will work on batch and near-real-time pipelines, collaborating with cross-functional teams to enhance data products and performance.

Qualifications

• Experience designing and implementing enterprise-scale data pipelines.
• Strong proficiency in Python for data engineering tasks.
• Experience with Lakehouse architectures and data lake solutions.

Responsibilities

• Build scalable ETL/ELT pipelines using Python and PySpark on Databricks.
• Develop batch and near-real-time data pipelines."
• Implement Bronze / Silver / Gold Medallion data architectures.
• Collaborate with teams to optimize data workflows and performance.

Skills

Python

Databricks

Apache Spark

PySpark

Tools

Databricks

Apache Spark

Python

Job description

Databricks is seeking a Databricks / Python Data Engineer to design, develop, and optimize enterprise-scale data pipelines and Lakehouse solutions using Python, Databricks, Apache Spark and PySpark. The role focuses on building scalable ETL/ELT processes and implementing medallion architectures for reliable data maturity.

The candidate will work on batch and near-real-time pipelines, collaborating with cross-functional teams to enhance data products and performance.