Senior Azure Data Engineer – Remote Data Lake & Pipelines

Eliassen Group; Boston (MA); Remote; USD 103,000 - 117,000; Full time

Job summary

Eliassen Group is seeking a Senior Azure Data Engineer to modernize an enterprise data platform by migrating workloads from on-prem Oracle/DataStage to a cloud Azure data lake, designing scalable pipelines with Databricks and Data Factory.

The role requires contributing to development and guiding data-architecture decisions, with strong focus on data quality, validation, and engineering best practices. Remote, contract-to-hire, W2 benefits package offered.

Qualifications

• Strong hands-on data engineering experience in enterprise environments.
• Experience with Azure Databricks and Azure Data Factory.
• Strong SQL and Python or PySpark development skills.
• Experience designing and developing batch data pipelines and ETL or ELT workflows.
• Experience implementing data-quality, data-validation, and reconciliation processes.
• Knowledge of medallion or lakehouse architecture, including Bronze, Silver, and Gold data layers.

Responsibilities

• Design, develop, test, and maintain data pipelines using Azure Databricks and Azure Data Factory.
• Support the migration of data and ETL workloads from an on-premises Oracle and DataStage environment to an Azure cloud data lake.
• Develop scalable ingestion, transformation, and data-processing solutions.
• Design and implement medallion architecture patterns across Bronze, Silver, and Gold data layers.
• Build data-quality, reconciliation, and validation checks directly into data pipelines and engineering workflows.
• Automate validation and testing to improve the reliability of data products.
• Help guide technical discussions and decisions related to data architecture, solution design, and engineering standards.
• Troubleshoot pipeline failures, data-quality issues, and performance problems.
• Collaborate with engineers, analysts, architects, and business stakeholders to translate requirements into practical data solutions.
• Participate in code reviews, documentation, testing, deployment, and continuous-improvement activities.
• Promote reusable development patterns and sound engineering practices across the team.

Skills

Azure Databricks

Azure Data Factory

SQL

Python/PySpark

ETL/ELT pipelines

Data quality checks

Medallion architecture

Data modeling

Git

Azure DevOps

CI/CD

Tools

Git

Azure DevOps

CI/CD tooling

Job description

Eliassen Group is seeking a Senior Azure Data Engineer to modernize an enterprise data platform by migrating workloads from on-prem Oracle/DataStage to a cloud Azure data lake, designing scalable pipelines with Databricks and Data Factory.

The role requires contributing to development and guiding data-architecture decisions, with strong focus on data quality, validation, and engineering best practices. Remote, contract-to-hire, W2 benefits package offered.