Azure Data Engineer: Pipelines, Lakes & Cloud Analytics

Machine Learning Technologies, LLC; Grand Rapids (MI); On-site; USD 120,000 - 160,000; Full time

Job summary

Machine Learning Technologies, LLC in the United States is seeking a data engineer to design, implement, and optimize Azure-based data pipelines and lakes. You will build ETL/ELT workflows, transform data with SQL, Spark, and Mapping Data Flows, and ensure secure, governed access to cloud data platforms.

Collaboration with analysts and stakeholders is essential for delivering scalable analytics solutions. The role focuses on migrating on-premises data to Azure, maintaining data architecture

Qualifications

• Expertise with Azure Data Factory for pipeline development and maintenance.
• Experience with Azure Data Lake Storage and enterprise-scale data lakes.
• Experience developing ETL/ELT workflows ingesting from multiple sources.
• Knowledge of Mapping Data Flows, SQL, and Spark for data transformation.
• Experience with incremental loads, CDC, and data validation.
• Ability to troubleshoot and optimize data pipelines for performance.
• Familiarity with data modeling, warehousing, and cloud data security/governance.
• Experience supporting on-premises data migration to Azure cloud environments.

Responsibilities

• Design, develop, and maintain data pipelines using Azure Data Factory.
• Build and manage enterprise-scale data lakes using Azure Data Lake Storage.
• Develop ETL/ELT workflows to ingest data from multiple sources (DBs, APIs, files, cloud apps).
• Implement data transformation processes using Mapping Data Flows, SQL, and Spark.
• Monitor, troubleshoot, and optimize data pipelines for reliability and performance.
• Configure incremental loads, CDC, and data validation processes.
• Collaborate with analysts, stakeholders, and developers to understand data requirements.
• Design data models and support data warehousing solutions.
• Implement security, governance, and access controls for cloud data platforms.
• Create technical documentation and maintain data architecture standards.
• Support migration of on-premises data solutions to Azure cloud.

Skills

Azure Data Factory

Azure Data Lake Storage

ETL/ELT

Data pipelines

SQL

Spark

CDC

Data modeling

Data warehousing

Security & governance

Tools

Mapping Data Flows

Job description

Machine Learning Technologies, LLC in the United States is seeking a data engineer to design, implement, and optimize Azure-based data pipelines and lakes. You will build ETL/ELT workflows, transform data with SQL, Spark, and Mapping Data Flows, and ensure secure, governed access to cloud data platforms.

Collaboration with analysts and stakeholders is essential for delivering scalable analytics solutions. The role focuses on migrating on-premises data to Azure, maintaining data architecture