Senior Data Engineer: Databricks & ADF ETL Lead

Ethereum Technologies; Sunnyvale (CA); Vor Ort; USD 91.000 - 124.000; Vollzeit

Zusammenfassung

Ethereum Technologies in Sunnyvale, TX seeks a Lead Data Engineer to design and optimize data pipelines using Databricks and Azure Data Factory (ADF). You will lead ETL/ELT processes, write SQL, and implement PySpark/Python transformations to support analytics initiatives.

Responsibilities include maintaining data pipelines, ensuring data quality, and collaborating with cross-functional teams to deliver reliable data solutions in a cloud-first environment.

Qualifikationen

• Experience with Databricks and Azure Data Factory (ADF).
• Strong understanding of ETL/ELT concepts.
• Good knowledge of SQL and data transformations.
• Basic experience with Python or PySpark.
• Understanding of Azure cloud services and data lake concepts.

Aufgaben

• Develop and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
• Work with Databricks for data processing and transformation.
• Write and optimize SQL queries for data extraction and transformation.
• Work with PySpark or Python for large-scale data processing.
• Integrate data from multiple sources into data warehouses or data lakes.
• Monitor, troubleshoot, and optimize data pipelines.
• Collaborate with teams to ensure data quality and reliability.

Kenntnisse

Databricks

Azure Data Factory

ETL/ELT concepts

SQL

Python / PySpark

Azure Data Lake / Azure cloud

Jobbeschreibung

Ethereum Technologies in Sunnyvale, TX seeks a Lead Data Engineer to design and optimize data pipelines using Databricks and Azure Data Factory (ADF). You will lead ETL/ELT processes, write SQL, and implement PySpark/Python transformations to support analytics initiatives.

Responsibilities include maintaining data pipelines, ensuring data quality, and collaborating with cross-functional teams to deliver reliable data solutions in a cloud-first environment.