Azure Data Engineer — Remote Databricks Pipelines Expert

Eliassen Group; Indianapolis (IN); À distance; USD 135 000 - 156 000; Temps partiel

Avantages offerts par ce poste

Medical benefits

Dental benefits

Vision benefits

401(k) with match

Life insurance

Résumé du poste

Eliassen Group is seeking an Azure Data Engineer for a remote, contract role focused on migrating an on‑prem Oracle/DataStage platform to an Azure data lake. You will design and implement scalable data pipelines with Databricks and Data Factory, and promote data through Bronze/Silver/Gold layers while ensuring data quality and governance across teams.

You will collaborate with engineers, architects, and business stakeholders to deliver robust data products, apply CI/CD, and contribute to

Qualifications

• Hands-on data engineering experience in enterprise environments.
• Demonstrated experience with Azure Databricks and Azure Data Factory.
• Strong SQL and Python or PySpark development skills.
• Experience designing and developing batch data pipelines and ETL/ELT workflows.
• Experience implementing data-quality, data-validation, and reconciliation processes.
• Knowledge of medallion or lakehouse architecture (Bronze/Silver/Gold).
• Experience with CI/CD and version control (Git).
• Awareness of healthcare/finance data and regulated environments is a plus.

Responsabilités

• Design, develop, test, and maintain data pipelines using Azure Databricks and Data Factory.
• Migrate on-premises Oracle/DataStage workloads to an Azure data lake.
• Create scalable ingestion, transformation, and data-processing solutions.
• Implement medallion architecture across Bronze, Silver, Gold layers.
• Embed data-quality checks and validation into pipelines and workflows.
• Automate testing and validation to improve data reliability.
• Troubleshoot pipeline failures and performance issues.
• Collaborate with engineers, analysts, and stakeholders to translate requirements.
• Participate in code reviews, documentation, and CI/CD practices.

Connaissances

SQL

Python / PySpark

ETL / ELT

Medallion Architecture

CI / CD

Git

Outils

Azure Databricks

Azure Data Factory

Delta Lake

Azure DevOps

Description du poste

Eliassen Group is seeking an Azure Data Engineer for a remote, contract role focused on migrating an on‑prem Oracle/DataStage platform to an Azure data lake. You will design and implement scalable data pipelines with Databricks and Data Factory, and promote data through Bronze/Silver/Gold layers while ensuring data quality and governance across teams.

You will collaborate with engineers, architects, and business stakeholders to deliver robust data products, apply CI/CD, and contribute to