Lead Data Engineer - Databricks & ADF ETL

Robotic Technologies Automation; Sunnyvale (CA); Presencial; USD 65 000 - 107 000; Tempo integral

Resumo da oferta

Robotic Technologies Automation in Sunnyvale is seeking a Lead Data Engineer to design, build, and optimize data pipelines using Databricks and Azure Data Factory (ADF). You will develop ETL/ELT workflows, write SQL, and leverage PySpark or Python for scalable data processing in a cloud environment.

The role demands collaboration with cross-functional teams to ensure data quality and reliability, with a focus on scalable data lake/warehouse integration and operational efficiency.

Qualificações

• Experience with Databricks and Azure Data Factory (ADF).
• Strong understanding of ETL/ELT concepts.
• Good knowledge of SQL and data transformations.
• Basic experience with Python or PySpark.

Responsabilidades

• Develop and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
• Work with Databricks for data processing and transformation.
• Write and optimize SQL queries for data extraction and transformation.
• Work with PySpark or Python for large-scale data processing.
• Integrate data from multiple sources into data warehouses or data lakes.
• Monitor, troubleshoot, and optimize data pipelines.
• Collaborate with teams to ensure data quality and reliability.

Conhecimentos

ETL/ELT concepts

SQL data transformations

Python scripting

Ferramentas

Databricks

Azure Data Factory

Python/PySpark

Descrição da oferta de emprego

Robotic Technologies Automation in Sunnyvale is seeking a Lead Data Engineer to design, build, and optimize data pipelines using Databricks and Azure Data Factory (ADF). You will develop ETL/ELT workflows, write SQL, and leverage PySpark or Python for scalable data processing in a cloud environment.

The role demands collaboration with cross-functional teams to ensure data quality and reliability, with a focus on scalable data lake/warehouse integration and operational efficiency.