Remote Senior Data Engineer - Azure Databricks ML Pipelines

Traffit; Województwo pomorskie; On-site; PLN 246,000 - 357,000; Full time

Benefits offered by this job

Remote work

Job summary

Traffit is seeking a skilled Data Engineer to join an international tech and data team on a 6-month engagement, starting in the first or second week of October. The role is remote and focuses on building data solutions for the insurance domain using Databricks and Spark.

You will design scalable data pipelines in Azure, transform and model data for ML and analytics, and collaborate with stakeholders while keeping code clean and tested.

Qualifications

• Fluent English (B2) is required.
• Commercial experience as a Data Engineer or in a similar data-focused technical role.
• Available to start in the first or second week of October.
• Available for a 6-month contract.

Responsibilities

• Design and develop scalable data pipelines using Azure, Databricks, Python and PySpark.
• Use Databricks AI tooling and Spark-based processing to extract and process data from typed and handwritten insurance documentation.
• Clean, validate, transform and model data for machine learning, analytics and BI use cases.
• Build reliable datasets that can support downstream reporting, analytical models and AI/ML solutions.
• Develop and maintain data transformations using Python and PySpark.
• Support end-to-end data engineering and analytics initiatives, from data ingestion through processing, quality validation and delivery.
• Apply clean-code principles, write maintainable and well-documented code, and contribute to code reviews.
• Create and maintain unit tests to ensure the quality, reliability and stability of data pipelines.
• Collaborate with data, analytics and business stakeholders to understand requirements and translate them into practical technical solutions.
• Identify data-quality issues and contribute to improvements in data governance, consistency and usability.

Skills

Fluent English

Tools

Microsoft Azure

Databricks

Python

PySpark

Spark

Job description

Traffit is seeking a skilled Data Engineer to join an international tech and data team on a 6-month engagement, starting in the first or second week of October. The role is remote and focuses on building data solutions for the insurance domain using Databricks and Spark.

You will design scalable data pipelines in Azure, transform and model data for ML and analytics, and collaborate with stakeholders while keeping code clean and tested.