Azure Data Engineer, Databricks Certified

Jobtailor; Deutschland; Vor Ort; EUR 80.000 - 120.000; Vollzeit

Zusammenfassung

Jobtailor is seeking a senior data engineer to design and operate scalable data solutions on the Microsoft Azure platform in a Germany-based role. You will own end-to-end data pipelines, from ingestion through storage to transformation, ensuring data quality and availability for analytics teams.

You will collaborate with data scientists and analysts, applying best practices in data governance and security while leveraging Azure Databricks, Data Factory, and Synapse to enable data-driven insights.

Qualifikationen

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• 5+ years of experience with Big Data technologies and cloud native data platforms.
• Strong knowledge of Microsoft Azure cloud services and data tooling.
• Proficient in SQL and SQL-based databases.
• Experience with batch and streaming data processing.

Aufgaben

• Design, develop, and implement scalable data solutions on Azure.
• Collaborate with cross-functional teams to gather data requirements.
• Build data ingestion pipelines and ensure data integrity.
• Perform data integration and transformation with quality controls.
• Utilize Azure SQL, Data Lake, and Blob Storage for storage and retrieval.
• Monitor pipelines and troubleshoot to maintain data availability.
• Implement data governance, privacy, and quality measures.
• Partner with data scientists to enable data-driven insights.

Kenntnisse

Azure data services

SQL

Data pipelines

Spark

Python

ETL

Data modeling

Data governance

Analytical thinking

Team collaboration

Ausbildung

Bachelor's degree in CS/Engineering

Tools

Azure Data Factory

Azure Databricks

Azure Data Lake

Azure Synapse

Snowflake

CosmosDB

Hadoop

SQL-based databases

Jobbeschreibung

Responsibilities

• Design, develop, and implement scalable and reliable data solutions on the Microsoft Azure platform.
• Collaborate with cross‑functional teams to gather and analyze data requirements.
• Design and implement data ingestion pipelines to collect data from various sources, ensuring data integrity and reliability.
• Perform data integration and transformation activities, ensuring data quality and consistency.
• Implement data storage and retrieval mechanisms, utilizing Azure services such as Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
• Monitor data pipelines and troubleshoot issues to ensure smooth data flow and availability.
• Implement data quality measures and data governance practices to ensure data accuracy, consistency, and privacy.
• Collaborate with data scientists and analysts to support their data needs and enable data‑driven insights.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 5+ years of experience with Big Data technologies
• Strong knowledge and experience with Microsoft Azure cloud platform
• Proficiency in SQL and experience with SQL-based database systems.
• Experience with batch and data streaming.
• Proficiency in data processing frameworks such as Apache Spark, Apache Hadoop, or cloud-native data processing services (Azure Data Lake, Azure Data factory, Azure Databricks, Azure Synapse, Snowflake, CosmosDB)
• Hands‑on experience with Azure data services, such as Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
• Experience using Azure Databricks in real-world scenarios.
• Experience with data integration and ETL (Extract, Transform, Load) processes.
• Strong analytical and problem-solving skills.
• Good understanding of data engineering principles and best practices.
• Experience with programming languages such as Python or Scala.
• Relevant certifications in Azure data services or data engineering are a plus.

Core Competencies

Demonstrates expertise in designing and implementing scalable data solutions on the Microsoft Azure platform, with a strong focus on data ingestion, integration, and quality assurance. Proficient in utilizing Azure data services and frameworks to support data-driven insights and ensure data integrity.