Lead Data Engineer (all genders)

Beiersdorf AG; Hamburg; Vor Ort; EUR 90.000 - 130.000; Vollzeit

Zusammenfassung

Beiersdorf AG in Hamburg is seeking a Lead Data Engineer to design, build and support data pipelines and databases within our Azure/Databricks Data & Analytics platform. You will balance business requirements with stable, scalable solutions and work with project leads, data scientists and cloud engineers across global projects.

You will mentor mid-level Data Engineers (no people management) and drive implementation of serverless components, observability, CI/CD and data governance, leveraging

Qualifikationen

• Bachelor’s or Master’s degree in a related field.
• Several years of Data Engineering or Software Development experience in cloud-native environments.
• Strong Azure expertise (Azure Data Factory, Databricks, Synapse, ADLS Gen2, Azure SQL).
• Proficiency in Python with PySpark; knowledge of Scala/Java/Go is a plus.

Aufgaben

• Design and implement data pipelines within Azure and Databricks tech stack.
• Develop advanced data processing solutions within Azure and Databricks.
• Conceptualize and implement relational and NoSQL databases in the cloud platform.
• Management/code practices using Azure DevOps.
• Oversee Data Lake House for structured, semi-structured, and unstructured data.
• Serve as primary technical liaison to Architecture Team and ensure standards.
• Maintain feedback loop between design and implementation.
• Mentor mid-level Data Engineers in the team.

Kenntnisse

Python

PySpark

Scala/Java/Go

Azure cloud

Databricks

CI/CD

Observability

Data governance

Multi-cloud

Ausbildung

Bachelor’s or Master’s degree in related field

Tools

Azure Data Factory

Databricks

Azure Synapse

ADLS Gen2

Kafka

Azure DevOps

Jobbeschreibung

Your Tasks

As a Lead Data Engineer you concept, design, implement and support data pipelines and databases within our Azure/Databricks Data & Analytics platform. You will always find the balance between individual requirements and stable solutions. You are open to new technologies and see them as an opportunity to enable even deeper insights into data. You will work closely with our project leads, reporting consultants, data scientist and cloud engineers. We work together on global projects in mixed project teams. It is important to us that everyone can contribute their experiences and concerns to discussions.

Your main tasks are:

• Design and develop data pipelines within Azure and Databricks tech stack
• Code advanced data processing solutions within Azure and Databricks tech stack
• Concept and implement relational and NoSQL database solutions within Azure and Databricks
• Development/code management using Azure DevOps
• Manage our Data Lake House which stores structured, semi-structured data and unstructured data
• Act as primary technical liaison to the Architecture Team, ensuring alignment with standards and guidelines
• Establish and maintain a productive feedback loop between architectural design and implementation
• Mentor technically (no people management) mid-level Data Engineers in the team

Your Profile

• Bachelor’s or Master’s degree in related field
• Several years of experience in Data Engineering or Software Development within cloud native environments.
• Deep operational expertise across the Azure ecosystem (Azure Data Factory, Databricks, Synapse, ADLS Gen2, Azure SQL).
• Strong knowledge of Azure serverless and integration services (Functions, Cosmos DB, Event Hubs/Service Bus).
• Solid implementation of observability practices (logging, metrics, distributed tracing) and secure secrets management.
• Expert-level proficiency in Python with strong PySpark experience.
• Proficiency in at least one compiled/typed language (Scala, Java, or Go).
• Ability to write clean, modular, and performance optimized code.
• Demonstrated capability in technology scouting and evaluating emerging trends for practical business value.
• Multi-cloud exposure (AWS/GCP) and experience with streaming architectures (Kafka, Spark Structured Streaming).
• Strong understanding of data governance concepts and CI/CD automation principles.

We also embrace diversity by valuing the uniqueness of each individual and being committed to equal opportunities for all.