Data Engineer - Databricks, Spark & Python

Inetum

Warszawa

On-site

PLN 180,000 - 260,000

Full time

5 days ago

Be an early applicant

Application generator

Stand out for this role — generate a tailored resume and cover letter in about a minute.

Get past ATS filters

Benefits offered by this job

Cafeteria benefits

Generous referral bonuses (up to PLN6,

Revenue sharing opportunities for new-

Job summary

Inetum Polska is seeking a Data Engineer to join an aviation-focused project that uses Azure and Databricks to build a scalable data platform for predictive maintenance and analytics on aircraft engines.

You will design and optimize data pipelines with Spark, Python, and SQL, implement CI/CD for Databricks apps, and collaborate with stakeholders to translate business needs into robust data solutions. This role emphasizes data quality and scalable processing for large-scale datasets.

Qualifications

Commercial experience in Data Engineering.
Hands-on experience with Databricks.
Experience in implementing CI/CD for Databricks applications.
Practical experience with Microsoft Azure data processing and storage services.
Strong experience with Apache Spark.
Good knowledge of SQL.
Hands-on development experience with Python.
Good understanding of Big Data processing concepts, standards, and tools.
Experience working with large-scale data processing pipelines and ETL/ELT processes.

Responsibilities

Design, develop, and maintain scalable data pipelines using Azure Databricks, Apache Spark, Python, and SQL.
Process and transform large volumes of operational and technical data supporting predictive maintenance and analytical use cases.
Develop and optimize ETL/ELT processes for efficient data ingestion, transformation, and storage.
Design and implement CI/CD processes for Databricks applications using Azure tooling, including Terraform and Databricks Asset Bundles (DABs).
Work with Azure data processing and storage services to build reliable and scalable data solutions.
Optimize Spark workloads and improve overall data processing performance.
Implement data validation and quality checks to ensure consistency, accuracy, and reliability of datasets.
Collaborate with Data Engineers and project stakeholders to translate business and data requirements into technical solutions.
Contribute to the continuous development, maintenance, and improvement of the existing data platform.
Troubleshoot data processing issues and improve the stability and performance of data pipelines.
Prepare and maintain technical documentation for implemented solutions and data workflows.