Azure data engineer in analytics platforms

i2d consulting; Warszawa; On-site; PLN 180,000 - 260,000; Full time

Job summary

i2d consulting in Poland is seeking an experienced Data Engineer to join the Data Services ART within the Client Data Analytics Platform. You will design, develop and operate scalable, secure data solutions to support analytics, AI and business-critical applications.

The ideal candidate has 8+ years in data or software engineering, hands-on Azure Databricks, ADLS, Event Hubs, Spark, Delta Lake, Terraform, and Azure DevOps. Strong English, collaboration, and automation focus are essential.

Qualifications

• 8+ years in data or software engineering or related role.
• Hands-on with large-scale data platforms or cloud data solutions.
• Strong SQL, Python, data modelling and data pipelines skills.
• Experience with Azure Databricks, ADLS, Event Hubs, Spark/Spark Streaming and Delta Lake.
• Understanding of lakehouse architectures and cloud-native platforms.
• Terraform and Infrastructure as Code experience.
• CI/CD pipelines with Azure DevOps.
• Knowledge of cloud security, observability, reliability, and scalability.
• Data governance, quality and production monitoring experience.
• Bachelor’s or Master’s in related field; excellent English and collaboration.

Responsibilities

• Design, develop, test and maintain scalable batch and streaming pipelines.
• Build cloud-native data solutions using Azure Databricks, ADLS, Event Hubs and Spark.
• Develop and manage lakehouse architectures and high-performance processing.
• Provision IaC using Terraform and manage CI/CD with Azure DevOps.
• Apply testing, versioning, docs and deployment automation best practices.
• Ensure data quality, metadata governance and production readiness.
• Troubleshoot issues, do root-cause analysis and implement fixes.
• Collaborate with architects, product owners, engineers and stakeholders.

Skills

SQL

Python

Data modeling

Data pipeline development

Education

Bachelor's or Master's in CS/SE/Data Eng

Tools

Azure Databricks

Azure Data Lake Storage

Azure Event Hubs

Apache Spark

Delta Lake

Terraform

Azure DevOps

Job description

Описание

We are looking for an experienced Data Engineer to join the Data Services ART within the Client Data Analytics Platform (ADAP). The role involves designing, developing and operating scalable, secure and reliable data solutions to support analytics, AI and business-critical applications. The ideal candidate will have strong hands-on experience in Azure cloud-native data engineering, Azure Databricks, large-scale data pipelines, infrastructure as code and automated CI/CD deployments.

Задачи

Design, develop, test and maintain scalable batch and streaming data pipelines, datasets, data models and data products. Build cloud-native data solutions using Azure Databricks, Azure Data Lake Storage (ADLS), Azure Event Hubs, Apache Spark and Delta Lake. Develop and manage lakehouse architectures and high-performance data processing solutions. Provision and manage cloud infrastructure using Terraform and Infrastructure as Code (IaC) practices. Build, maintain and optimize CI/CD pipelines using Azure DevOps. Apply software engineering best practices, including automated testing, version control, code quality, documentation and deployment automation. Implement monitoring, observability and operational controls to improve reliability, scalability, performance and cost efficiency. Ensure data quality, metadata management, governance, security and production readiness. Troubleshoot production issues, perform root-cause analysis and implement preventive measures. Collaborate with architects, product owners, engineers and business stakeholders to translate requirements into scalable technical solutions. Participate in agile ceremonies, code reviews, proof of concepts, knowledge sharing and continuous platform improvement.

Требования

8+ years of experience in data engineering, software engineering or a related technical role. Strong hands-on experience designing, building and operating large-scale data platforms or cloud-based data solutions. Strong proficiency in SQL, Python, data modelling and data pipeline development. Hands-on experience with Azure Databricks, Azure Data Lake Storage (ADLS), Azure Event Hubs, Apache Spark/Spark Streaming and Delta Lake. Strong understanding of lakehouse architectures and cloud-native data platforms. Practical experience with Terraform and Infrastructure as Code (IaC). Experience developing and managing CI/CD pipelines using Azure DevOps. Good understanding of cloud security, observability, reliability, scalability and operational best practices. Experience implementing data quality, governance and production monitoring practices. Bachelor’s or Master’s degree in Computer Science, Software Engineering, Data Engineering or a related discipline. Excellent communication and collaboration skills. Fluent in English, with the ability to work effectively in cross-functional teams. Proactive, ownership-driven and capable of solving complex technical problems. Strong commitment to automation, continuous improvement and engineering excellence. Immediate joiners with a valid work permit for the country of employment will be considered.

Будет плюсом

Experience with Rust; Exposure to Google Cloud Platform (GCP); Knowledge of Kubernetes, containers and containerisation technologies; Experience with data visualisation tools; Understanding of data governance, data management and data quality frameworks

Условия

Employment Type: Contract Work Mode: Onsite Location: Poland or Hungary