Senior Azure Data Engineer - Cloud Data Lake Architect

Eliassen Group; San Juan (PR); Remote; USD 103,000 - 117,000; Full time

Benefits offered by this job

Medical, Dental, Vision benefits

401k with company matching

Life insurance

Job summary

Senior Azure Data Engineer at Eliassen Group will lead the modernization of an enterprise data platform by migrating data and workloads from on-prem Oracle/IBM DataStage to an Azure-based data lake architecture. You will design, build, and optimize scalable data pipelines using Azure Databricks and Azure Data Factory, while guiding architecture discussions and engineering best practices.

The role emphasizes hands-on development and collaboration with engineers, architects, and business partners,

Qualifications

• Hands-on data engineering in enterprise environments.
• Experience with Azure Databricks & Azure Data Factory.
• Strong SQL and Python/PySpark development skills.
• Experience designing batch data pipelines and ETL/ELT workflows.
• Knowledge of medallion/lakehouse architecture (Bronze/Silver/Gold).
• Experience with relational databases and large datasets.
• Familiarity with CI/CD and automated deployment.

Responsibilities

• Design, develop, test, and maintain data pipelines using Azure Databricks and Azure Data Factory.
• Migrate on-premises data/workloads to Azure cloud data lake.
• Develop scalable ingestion, transformation and processing solutions.
• Implement medallion architecture across data layers.
• Add data-quality checks within pipelines and workflows.
• Automate validation and testing to improve reliability.
• Guide data architecture decisions and engineering standards.
• Troubleshoot pipeline failures and data-quality issues.
• Collaborate with engineers, analysts, and stakeholders.
• Participate in code reviews, documentation, and deployment.

Skills

Azure Databricks

Azure Data Factory

SQL

Python

PySpark

Data pipelines

Medallion architecture

Git

CI/CD

Tools

Git

Azure DevOps

Delta Lake

Databricks features

Job description

Senior Azure Data Engineer at Eliassen Group will lead the modernization of an enterprise data platform by migrating data and workloads from on-prem Oracle/IBM DataStage to an Azure-based data lake architecture. You will design, build, and optimize scalable data pipelines using Azure Databricks and Azure Data Factory, while guiding architecture discussions and engineering best practices.

The role emphasizes hands-on development and collaboration with engineers, architects, and business partners,