Azure Data Architect: Build Scalable Cloud Data Lakes

Eliassen Group; Saint Paul (MN); Remote; USD 90,000 - 103,000; Full time

Job summary

Eliassen Group is seeking an Azure Data Engineer to modernize an enterprise data platform by migrating data and workloads from an on-premises Oracle data warehouse and IBM DataStage to a cloud-based Azure data lake architecture.

You will design, build, and optimize scalable data pipelines using Azure Databricks and Azure Data Factory, implement data-quality checks, and promote data through medallion architecture layers.

Qualifications

• Hands-on data engineering in large enterprise environments.
• Strong experience with Azure Databricks and Azure Data Factory.
• Proficient in SQL and Python or PySpark.
• Experience building batch and ETL/ELT pipelines.
• Knowledge of medallion or lakehouse architectures.
• Experience with relational databases and large datasets.
• Experience with Git and CI/CD.
• Healthcare/finance domain knowledge preferred.

Responsibilities

• Design, develop, test, and maintain data pipelines using Azure Databricks and Azure Data Factory.
• Support migration of data and ETL workloads to an Azure cloud data lake.
• Develop scalable ingestion, transformation, and data-processing solutions.
• Design and implement medallion architecture across Bronze, Silver, and Gold layers.
• Build data-quality, reconciliation, and validation checks into pipelines.
• Automate validation and testing to improve data product reliability.
• Troubleshoot pipeline failures, data-quality issues, and performance problems.
• Collaborate with engineers, analysts, architects, and stakeholders to translate requirements.
• Participate in code reviews, documentation, testing, deployment, and continuous-improvement.
• Promote reusable development patterns and sound engineering practices.

Skills

Azure Databricks

Azure Data Factory

SQL

Python / PySpark

Data pipeline design

Medallion architecture

ETL / ELT

CI/CD practices

Git

Collaborative skills

Tools

Azure DevOps

Azure Databricks tooling

CI/CD tooling

Job description

Eliassen Group is seeking an Azure Data Engineer to modernize an enterprise data platform by migrating data and workloads from an on-premises Oracle data warehouse and IBM DataStage to a cloud-based Azure data lake architecture.

You will design, build, and optimize scalable data pipelines using Azure Databricks and Azure Data Factory, implement data-quality checks, and promote data through medallion architecture layers.