Data Engineer – Azure + PySpark Pipelines

Mercedes-Benz.io GmbH; Lisboa; Hybrid; EUR 55,000 - 75,000; Full time

Benefits offered by this job

Health insurance

Life insurance

Open source software

Flexible hybrid work model

International trainings and conference

Job summary

Mercedes-Benz.io is seeking a Data Engineer to build and maintain data pipelines powering analytics and decision-making across the organization. You will design scalable data infrastructure, work with Spark and Azure services, and collaborate with analysts to ensure high-quality data for reporting and ML workloads.

You will contribute to governance, monitoring, and production deployment while advancing platform performance and reliability in a fast-paced, hybrid environment.

Qualifications

• 3-5 years of hands-on software engineering experience building, maintaining, and optimizing data pipelines.
• Experience with OOP, TDD, CI/CD and version control practices.
• Strong PySpark and SQL proficiency with knowledge of Spark internals.
• Strong Python skills for data processing, automation and scalable pipelines.
• Expertise with Azure cloud tech: Databricks, Data Factory, Azure DevOps, Data Lake Storage.
• Solid experience in dimensional data modelling and data warehousing.
• Strong analytical and problem-solving skills; able to troubleshoot data challenges.
• Excellent communication and ability to work with technical and non-technical stakeholders.
• Collaborative mindset; thrive in fast-paced team environments.

Responsibilities

• Design and build scalable data pipelines for cleaning, integrating, and transforming large datasets.
• Create data processing infrastructure ensuring high-quality data availability for analysts and data scientists.
• Leverage Azure services including Data Factory, Databricks, and Data Lake Storage to build robust data solutions.
• Develop dimensional data models and maintain data warehousing for analytics and reporting needs.
• Write efficient Python and SQL code for large-scale data processing.
• Work with Apache Spark to develop and optimize PySpark code and SQL queries.
• Collaborate with data analysts and stakeholders to understand business requirements.
• Ensure data governance by monitoring data quality and implementing best practices.
• Support deployment, monitoring, and maintenance of ML pipelines and data solutions in production.
• Continuously improve platform performance, scalability, and reliability through automation.

Skills

3-5 years exp

OOP/TDD/CI-CD

PySpark

Python

Azure Databricks

Data Factory

Data Lake Storage

Dimensional modelling

Analytical skills

Communication

Collaboration

Tools

Kubernetes

Docker

Job description

Mercedes-Benz.io is seeking a Data Engineer to build and maintain data pipelines powering analytics and decision-making across the organization. You will design scalable data infrastructure, work with Spark and Azure services, and collaborate with analysts to ensure high-quality data for reporting and ML workloads.

You will contribute to governance, monitoring, and production deployment while advancing platform performance and reliability in a fast-paced, hybrid environment.