Data Engineer Specialist - Ingest & Analytics with PySpark

Generali Employee Benefits - GEB; Assago; In loco; EUR 35.000 - 45.000; Tempo pieno

Descrizione del lavoro

Redion Employee Benefits, based in Assago, Italy, seeks a Data Engineer Specialist to manage data ingestion pipelines using PySpark and Databricks. You will implement data quality checks, support CI/CD in Azure DevOps, and maintain Jira/Confluence docs while engaging with business units.

The role emphasizes translating business needs into technical specs, assisting in training, and improving systems. The position is permanent, with a base salary in the 35k–45k EUR range.

Competenze

• Minimum 3 years of experience in data engineering projects, ideally in insurance or financial services.
• Fluent in English; additional languages are a plus.
• Hands-on experience with Python (PySpark) and Databricks.
• PostgreSQL, REST and Graph APIs, and Azure Data Services.
• Familiarity with Medallion architecture (Bronze, Silver, Gold) and Delta Lake.
• Basic understanding of Unity Catalog for metadata and access control.
• Exposure to Git, CI/CD practices (Azure DevOps), and optionally Terraform.
• Comfortable in Agile teams using Jira and Confluence.
• Degree in computer science, engineering, or IT-related fields.

Mansioni

• Develop ingestion pipelines using PySpark and Databricks based on provided design specifications.
• Implement data quality validation rules within the established framework.
• Support CI/CD integration by using pre-configured Azure DevOps pipelines.
• Maintain documentation in Jira and Confluence, including updates to training and enablement content as features evolve.
• Support business engagement by attending meetings with key departments and assisting in training sessions when needed.
• Translate business requests into clear technical specifications to support development and testing.
• Provide operational support for integrations, tools, and third-party applications.

Conoscenze

Python

PySpark

Databricks

PostgreSQL

REST APIs

Graph APIs

Azure Data Services

Delta Lake

Unity Catalog

Git

Azure DevOps

Terraform

Jira

Confluence

Agile

English

Formazione

Bachelor’s or Master’s in computer science/engineering/IT

Strumenti

Azure DevOps

CI/CD

Descrizione del lavoro

Redion Employee Benefits, based in Assago, Italy, seeks a Data Engineer Specialist to manage data ingestion pipelines using PySpark and Databricks. You will implement data quality checks, support CI/CD in Azure DevOps, and maintain Jira/Confluence docs while engaging with business units.

The role emphasizes translating business needs into technical specs, assisting in training, and improving systems. The position is permanent, with a base salary in the 35k–45k EUR range.