Azure Databricks Data Engineer: Scale & Optimize Pipelines

Adastra Corp; United States; Remote; USD 120,000 - 160,000; Full time

Job summary

Adastra Corp is seeking an Azure Databricks Data Engineer to design, develop, and optimize enterprise-scale data platforms on Microsoft Azure. You will lead the implementation of scalable data pipelines, data models, and Lakehouse architectures, collaborating with stakeholders and mentoring engineers.

The role emphasizes high-performance, production-grade solutions with a focus on governance and security. Experience with Unity Catalog, Spark, PySpark, Azure Data Factory, and Azure Data Lake is

Qualifications

• Hands-on experience with Azure Databricks, Spark/PySpark, Azure Data Factory and Azure Data Lake.
• Design and build scalable data pipelines, data models and Lakehouse architectures.
• Strong collaboration with stakeholders and mentoring engineers.

Responsibilities

• Design and build scalable data platforms using Azure Databricks.
• Develop and optimize ETL/ELT pipelines using PySpark and Azure Data Factory.
• Design Data Lake, Data Warehouse and Data Mesh solutions.
• Optimize Spark jobs for performance and cost.
• Build CI/CD pipelines using Azure DevOps/GitHub/Terraform.
• Implement Unity Catalog, data governance and security best practices.
• Design data models for analytics and reporting.
• Collaborate with cross-functional teams and mentor junior engineers.

Skills

Azure Databricks

Spark/PySpark

Azure Data Factory

Azure Data Lake

Data Modeling

CI/CD Pipelines

Data Governance

Tools

Unity Catalog

Terraform

Azure DevOps

GitHub

Job description

Adastra Corp is seeking an Azure Databricks Data Engineer to design, develop, and optimize enterprise-scale data platforms on Microsoft Azure. You will lead the implementation of scalable data pipelines, data models, and Lakehouse architectures, collaborating with stakeholders and mentoring engineers.

The role emphasizes high-performance, production-grade solutions with a focus on governance and security. Experience with Unity Catalog, Spark, PySpark, Azure Data Factory, and Azure Data Lake is