Lead Data Engineer

Jobtailor; Deutschland; Vor Ort; EUR 90.000 - 140.000; Vollzeit

Zusammenfassung

Jobtailor seeks a Senior Data Engineer to support a U.S.-based client and build scalable data solutions. You will develop PySpark pipelines, collaborate with ML engineers, software developers, analytics engineers, and product managers, and translate business needs into highly available data systems.

Responsibilities include maintaining pipelines, data migration using Azure services, and leveraging Power BI for insights.

Qualifikationen

• 8+ years of experience translating business requirements into scalable data solutions.
• 3+ years building data pipelines with PySpark.
• Proven SQL experience.
• Advanced English (B2 or higher).
• Databricks certification preferred.
• Experience with Azure data services for migration and processing.
• Knowledge of SAP HANA.
• Knowledge of Power BI.
• DevOps / CI/CD and cloud migration experience.

Aufgaben

• Develop and optimize data pipelines and migrations.
• Collaborate with ML engineers, software developers, analytics engineers, and product managers.
• Translate business requirements into reliable data solutions.
• Maintain and improve data pipelines and systems.
• Support Azure-based data migrations and implementation.

Kenntnisse

PySpark

SQL

Power BI

DevOps

CI/CD

Azure cloud

Data pipelines

Data migration

Azure Synapse

Ausbildung

Databricks Certification

Tools

Azure Data Factory

Azure Data Lake Storage (ADLS)

Synapse

Azure SQL DB

Fabric

Azure Event Hub

IoT Hub

Stream Analytics

Cosmos DB

SAP HANA

Jobbeschreibung

Overview

Work with one of our U.S.-based clients.

Develop and implement data pipelines using PySpark.

Collaborate with cross-functional teams including machine learning engineers, software developers, analytics engineers, and product managers.

Translate business requirements into highly available data solutions.

Maintain and improve existing data pipelines and systems.

Requirements

• 8+ years of total experience collaborating with cross-functional teams, including machine learning engineers, software developers, analytics engineers, and product managers to translate business requirements into highly available data solutions
• 3+ years of experience developing and implementing data pipelines using PySpark
• Proven experience with SQL
• Advanced written and spoken English (B2 or C1/C2)
• Databricks Certification (nice to have)
• Experience providing practical direction within Azure native services, implementing data migration and data processing using Azure services such as ADLS, Azure Data Factory, Synapse, Azure SQL DB, Fabric
• Experience with Azure Event Hub, IoT Hub, Azure Stream Analytics, Azure Analysis Services, Cosmos DB
• Knowledge of SAP HANA
• Intermediate knowledge of Power BI
• Knowledge of DevOps and CI/CD deployments, cloud migration methodologies and processes

ATS Optimization Keywords

Below are skills and terms extracted directly from this job posting to improve Applicant Tracking System (ATS) visibility. This feature helps candidates tailor their applications more effectively.

Hard Skills

• PySpark
• SQL
• data pipelines
• data migration
• data processing
• Power BI
• DevOps
• CI/CD
• Azure services
• Cloud migration methodologies

Soft Skills

• Collaboration
• Communication
• Translation of business requirements
• Practical direction

Certifications & Qualifications

• Databricks Certification