Remote Azure Data Engineer – Databricks Data Lakes

Eliassen Group; Santa Fe (NM); Remote; USD 90,000 - 103,000; Full time

Benefits offered by this job

Medical benefits

Dental benefits

Vision benefits

401k with match

Life insurance

Job summary

Eliassen Group seeks an Azure Data Engineer to modernize an enterprise data platform by migrating from on-prem Oracle/DataStage to an Azure-based data lake architecture. You will design, build, and optimize scalable data pipelines using Azure Databricks and Azure Data Factory, implement data quality, and promote data through Bronze, Silver, and Gold layers.

You will collaborate with engineers, analysts, and business partners to deliver reliable data products, with strong focus on Azure

Qualifications

• Hands-on data engineering in enterprise environments.
• Experience with Azure Databricks and Azure Data Factory.
• Strong SQL and Python/PySpark development skills.
• Design and develop batch data pipelines and ETL/ELT workflows.
• Implement data-quality, data-validation, and reconciliation processes.
• Knowledge of medallion or lakehouse architecture (Bronze/Silver/Gold).
• Experience with relational databases and large datasets.
• Understanding of data modeling, schema design, partitioning, and performance optimization.
• Experience with Git or another VCS.
• Experience with CI/CD and automated deployment practices.
• Analytical, troubleshooting and problem-solving skills.
• Ability to contribute to development while guiding teammates.
• Excellent communication and collaboration skills.
• Preferred: healthcare/finance data, Azure DevOps, Agile/Scrum, on-prem to Azure migrations.

Responsibilities

• Design, develop, test, and maintain data pipelines using Azure Databricks and Azure Data Factory.
• Support migration of on-premises data and ETL workloads to Azure cloud data lake.
• Develop scalable ingestion, transformation, and data-processing solutions.
• Implement medallion architecture across Bronze, Silver, Gold layers.
• Build data-quality and validation checks into pipelines and workflows.
• Automate validation and testing to improve data product reliability.
• Troubleshoot pipeline failures and data-quality issues.
• Collaborate with engineers, analysts, architects, and stakeholders to translate requirements.
• Participate in code reviews, documentation, testing, deployment, and continuous improvement.
• Promote reusable patterns and best practices across the team.

Skills

SQL

Python

PySpark

Data pipelines

Azure Databricks

Azure Data Factory

ETL/ELT

Medallion architecture

Git

CI/CD

Analytics

Problem solving

Communication

Tools

Azure Databricks

Azure Data Factory

Oracle

DataStage

SQL databases

Job description

Eliassen Group seeks an Azure Data Engineer to modernize an enterprise data platform by migrating from on-prem Oracle/DataStage to an Azure-based data lake architecture. You will design, build, and optimize scalable data pipelines using Azure Databricks and Azure Data Factory, implement data quality, and promote data through Bronze, Silver, and Gold layers.

You will collaborate with engineers, analysts, and business partners to deliver reliable data products, with strong focus on Azure