Azure Data Engineer — Databricks & Data Factory Expert

SCIGON; Camden (NJ); On-site; USD 98,000 - 130,000; Full time

Job summary

SCIGON is seeking an experienced Azure Data Engineer to design and implement scalable data solutions on Microsoft Azure and Databricks. You will build robust ETL/ELT pipelines, optimize data flows, and collaborate with analytics teams to empower reporting and insights.

The ideal candidate has hands-on experience with Azure Databricks, ADF, Delta Lake, Spark, and Medallion Architecture, as well as Terraform and GitHub Actions for CI/CD and IaC.

Qualifications

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field.
• 5+ years of experience in Data Engineering or related technical roles.
• Strong experience with Azure Databricks, Azure Data Factory, ADLS, Apache Spark, and Delta Lake.
• Experience implementing Medallion Data Architecture.
• Hands-on Terraform for infrastructure automation.
• Experience building CI/CD pipelines using GitHub Actions.
• Experience in Agile environments.
• Excellent analytical and problem-solving skills.

Responsibilities

• Design, develop, and maintain scalable data platforms on Microsoft Azure.
• Build and optimize ETL/ELT data pipelines using Azure Data Factory and Azure Databricks.
• Develop data transformation processes leveraging Apache Spark and Delta Lake technologies.
• Implement and maintain Medallion Architecture to support analytics and reporting needs.
• Collaborate with business stakeholders, analysts, data scientists, and application teams.
• Design efficient data models that support reporting and BI initiatives.
• Monitor, troubleshoot, and optimize data pipelines for performance and reliability.
• Develop and maintain CI/CD pipelines using GitHub Actions.
• Provision and manage cloud infrastructure using Terraform and IaC best practices.
• Ensure data quality, governance, security, and compliance standards.
• Provide production support and resolve data integration issues.

Skills

Azure Databricks

Azure Data Factory

ADLS

Apache Spark

Delta Lake

SQL & data modeling

Medallion Architecture

Education

Bachelor's degree in Computer Science, Information Systems, Engineering, or related field

Tools

Terraform

GitHub Actions

Job description

SCIGON is seeking an experienced Azure Data Engineer to design and implement scalable data solutions on Microsoft Azure and Databricks. You will build robust ETL/ELT pipelines, optimize data flows, and collaborate with analytics teams to empower reporting and insights.

The ideal candidate has hands-on experience with Azure Databricks, ADF, Delta Lake, Spark, and Medallion Architecture, as well as Terraform and GitHub Actions for CI/CD and IaC.