Senior Databricks Data Engineer - Azure & Orchestration

Acuity Analytics; Porto; On-site; EUR 50,000 - 75,000; Full time

Job summary

Acuity Analytics is seeking a skilled Data Engineer in Porto to design, build, and manage scalable data solutions in a cloud-native environment. You will focus on automation, performance, and collaboration across teams to deliver end-to-end data pipelines using Databricks and Azure.

Responsibilities include creating Databricks Workflows, deploying PyFunc models, and ensuring secure configuration with Azure Key Vault, all while integrating Unity Catalog along the data lakehouse pattern.

Qualifications

• Experience with Databricks, Azure Cloud, and data orchestration.
• Design, build scalable data solutions in a cloud-native environment.
• Strong Python skills for data engineering and automation.

Responsibilities

• Build and manage Databricks Workflows for large-scale data orchestration.
• Develop and deploy PyFunc models into production pipelines.
• Implement secure secrets and configuration management using Azure Key Vault.
• Design and optimize ETL/ELT pipelines following best practices.
• Govern and manage data assets using Unity Catalog and Data Lakehouse patterns.
• Implement messaging and event-driven workflows with Azure Service Bus.
• Collaborate with architects, analysts, and data scientists to deliver end-to-end solutions.

Skills

Python

Production-ready code

CI/CD

Tools

Databricks

Azure

Unity Catalog

Job description

Acuity Analytics is seeking a skilled Data Engineer in Porto to design, build, and manage scalable data solutions in a cloud-native environment. You will focus on automation, performance, and collaboration across teams to deliver end-to-end data pipelines using Databricks and Azure.

Responsibilities include creating Databricks Workflows, deploying PyFunc models, and ensuring secure configuration with Azure Key Vault, all while integrating Unity Catalog along the data lakehouse pattern.