Azure Data Platform Engineer – Lakehouse Pipelines (Remote)

Eliassen Group; Sacramento (CA); Remote; USD 90,000 - 103,000; Full time

Benefits offered by this job

Medical, Dental, and Vision benefits

401k with company matching

Life insurance

Job summary

Eliassen Group is seeking an Azure Data Engineer to modernize an enterprise data platform by migrating workloads from on-prem Oracle/DataStage to Azure data lake architecture.

You will design, build, and optimize scalable data pipelines with Azure Databricks and Data Factory, implementing data quality and medallion patterns. Collaboration with engineers and business partners is essential.

Qualifications

• Hands-on data engineering experience in large-scale environments.
• Proven experience with Azure Databricks and Azure Data Factory.
• Strong SQL and Python or PySpark development skills.
• Experience designing batch ETL/ELT workflows.
• Knowledge of medallion or lakehouse patterns (Bronze/Silver/Gold).
• Experience with relational databases and large datasets.
• Familiarity with Git and CI/CD practices.
• Ability to mentor teammates and contribute technically.

Responsibilities

• Design, develop, test, and maintain data pipelines using Azure Databricks and Data Factory.
• Migrate on-prem data and ETL workloads to an Azure cloud data lake.
• Develop scalable ingestion, transformation, and data-processing solutions.
• Implement medallion architecture across Bronze, Silver, Gold layers.
• Embed data quality, reconciliation, and validation in pipelines.
• Automate validation and testing to improve reliability.
• Troubleshoot pipeline failures and data-quality issues.
• Collaborate with engineers, analysts, and stakeholders to translate requirements.
• Participate in code reviews, documentation, and deployment activities.
• Promote reusable patterns and best practices across the team.

Skills

Azure Databricks

SQL

Python/PySpark

ETL/ELT

Data pipelines

CI/CD

Git

Medallion architecture

Data quality

Collaborative skills

Tools

Azure Databricks

Azure Data Factory

Job description

Eliassen Group is seeking an Azure Data Engineer to modernize an enterprise data platform by migrating workloads from on-prem Oracle/DataStage to Azure data lake architecture.

You will design, build, and optimize scalable data pipelines with Azure Databricks and Data Factory, implementing data quality and medallion patterns. Collaboration with engineers and business partners is essential.