Remote Azure Data Engineer - Cloud Data Lake Pipelines

Eliassen Group; Phoenix (AZ); Remote; USD 90,000 - 103,000; Full time

Job summary

Eliassen Group is seeking an Azure Data Engineer to modernize an enterprise data platform by migrating an on-prem Oracle/DataStage environment to a cloud-based Azure data lake. You will design, build, and optimize scalable data pipelines using Azure Databricks and Azure Data Factory, implementing data quality and validation while promoting data through medallion layers.

Collaborate with engineers, analysts, and business partners to deliver reliable, maintainable data products.

Qualifications

• Hands-on data engineering in enterprise environments.
• Experience with Azure Databricks and Azure Data Factory.
• Strong SQL and Python or PySpark development skills.
• Experience designing and developing batch data pipelines and ETL/ELT workflows.
• Experience implementing data-quality, data-validation, and reconciliation processes.
• Knowledge of medallion or lakehouse architecture (Bronze/Silver/Gold).
• Experience with relational databases and large, complex datasets.
• Understanding of data modeling, partitioning, and performance optimization.
• Experience with version control (Git) and CI/CD practices.

Responsibilities

• Design, develop, test, and maintain data pipelines using Azure Databricks and Azure Data Factory.
• Support migration of data and ETL workloads from on-premises Oracle/DataStage to Azure cloud data lake.
• Develop scalable ingestion, transformation, and data-processing solutions.
• Design and implement medallion architecture across Bronze, Silver, Gold layers.
• Build data-quality, reconciliation, and validation checks into pipelines and workflows.
• Automate validation and testing to improve reliability of data products.
• Troubleshoot pipeline failures and data-quality issues.

Skills

Azure Databricks

Python

SQL

ETL/ELT

Data modeling

CI/CD

Tools

Azure Data Factory

Git

Job description

Eliassen Group is seeking an Azure Data Engineer to modernize an enterprise data platform by migrating an on-prem Oracle/DataStage environment to a cloud-based Azure data lake. You will design, build, and optimize scalable data pipelines using Azure Databricks and Azure Data Factory, implementing data quality and validation while promoting data through medallion layers.

Collaborate with engineers, analysts, and business partners to deliver reliable, maintainable data products.