Azure Data Engineer: Databricks & ADF ETL Mastery

MACHINE LEARNING TECHNOLOGIES LLC; Atlanta (GA); Vor Ort; USD 110.000 - 150.000; Vollzeit

Zusammenfassung

MACHINE LEARNING TECHNOLOGIES LLC in Atlanta, GA is seeking a Data Engineer with Databricks and Azure Data Factory to design, build, and optimize data pipelines in support of government-related data workloads.

You will collaborate with data teams to ensure data quality, implement ETL/ELT processes, and apply PySpark or Python for large-scale processing on Azure data lakes and warehouses.

Qualifikationen

• Experience with Databricks and Azure Data Factory (ADF).
• Strong understanding of ETL/ELT concepts.
• Good knowledge of SQL and data transformations.
• Basic experience with Python or PySpark.
• Understanding of Azure cloud services and data lake concepts.

Aufgaben

• Develop and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
• Work with Databricks for data processing and transformation.
• Write and optimize SQL queries for data extraction and transformation.
• Work with PySpark or Python for large-scale data processing.
• Integrate data from multiple sources into data warehouses or data lakes.
• Monitor, troubleshoot, and optimize data pipelines.
• Collaborate with teams to ensure data quality and reliability.

Kenntnisse

Databricks

Azure Data Factory (ADF)

ETL/ELT concepts

SQL

Python / PySpark

Azure cloud services

Tools

Databricks

Azure Data Factory (ADF)

Jobbeschreibung

MACHINE LEARNING TECHNOLOGIES LLC in Atlanta, GA is seeking a Data Engineer with Databricks and Azure Data Factory to design, build, and optimize data pipelines in support of government-related data workloads.

You will collaborate with data teams to ensure data quality, implement ETL/ELT processes, and apply PySpark or Python for large-scale processing on Azure data lakes and warehouses.