Data Engineer | Databricks & Azure Data Factory

MACHINE LEARNING TECHNOLOGIES LLC; Plano (TX); Presencial; USD 118 000 - 136 000; Tempo integral

Resumo da oferta

Machine Learning Technologies LLC in Plano, TX is seeking a Data Engineer to design, develop, and optimize data pipelines using Databricks and Azure Data Factory. You will build ETL/ELT processes, write SQL queries, and work with PySpark to handle large-scale data.

The role focuses on data quality, reliability, and collaboration with analytics teams to deliver robust data assets for reporting and analytics in a cloud-based environment.

Qualificações

• ETL/ELT concepts and data integration expertise.
• SQL proficiency for data extraction and transformation.
• Basic Python or PySpark for large-scale data processing.

Responsabilidades

• Develop and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
• Work with Databricks for data processing and transformation.
• Write and optimize SQL queries for data extraction and transformation.
• Utilize PySpark or Python for large-scale data processing tasks.
• Integrate data from multiple sources into data warehouses or data lakes.
• Monitor, troubleshoot, and optimize data pipelines.
• Collaborate with teams to ensure data quality and reliability.

Ferramentas

Databricks

Azure Data Factory (ADF)

Descrição da oferta de emprego

Machine Learning Technologies LLC in Plano, TX is seeking a Data Engineer to design, develop, and optimize data pipelines using Databricks and Azure Data Factory. You will build ETL/ELT processes, write SQL queries, and work with PySpark to handle large-scale data.

The role focuses on data quality, reliability, and collaboration with analytics teams to deliver robust data assets for reporting and analytics in a cloud-based environment.