Azure Data Engineer - ETL/ELT & Data Lake Specialist

MACHINE LEARNING TECHNOLOGIES LLC; Atlanta (GA); On-site; USD 91,000 - 129,000; Full time

Job summary

MACHINE LEARNING TECHNOLOGIES LLC in Atlanta, GA is seeking an experienced Azure Data Engineer to design and maintain enterprise data pipelines on Azure. You will work with large datasets and cloud-based services to support BI and analytics initiatives.

The role involves building ETL/ELT pipelines with Azure Data Factory, leveraging Azure Databricks and Data Lake Storage, optimizing SQL queries, and ensuring data quality. Collaboration with analytics teams is essential.

Qualifications

• Proficient in Python and SQL with hands-on data engineering experience.
• Strong knowledge of PySpark and Spark-based processing.
• Experience designing ETL/ELT pipelines in cloud environments.
• Familiar with cloud data architecture and data warehousing concepts.
• Experience with Azure Data Factory and Azure data services.

Responsibilities

• Design and develop ETL/ELT data pipelines.
• Build data integration solutions using Azure Data Factory.
• Work with Azure Databricks and Azure Data Lake.
• Optimize data processing workflows and SQL queries.
• Implement data quality and validation processes.
• Collaborate with analytics and BI teams.

Skills

Python

SQL

PySpark

Cloud data architecture

ETL concepts

Tools

Azure Data Factory

Azure Databricks

Azure Data Lake Storage

Job description

MACHINE LEARNING TECHNOLOGIES LLC in Atlanta, GA is seeking an experienced Azure Data Engineer to design and maintain enterprise data pipelines on Azure. You will work with large datasets and cloud-based services to support BI and analytics initiatives.

The role involves building ETL/ELT pipelines with Azure Data Factory, leveraging Azure Databricks and Data Lake Storage, optimizing SQL queries, and ensuring data quality. Collaboration with analytics teams is essential.