Data Engineer (Databricks) (f/m/d)

HCLTech Germany; Frankfurt; Sur place; EUR 85 000 - 125 000; Plein temps

Résumé du poste

HCLTech Germany is seeking an experienced Data Engineer (Databricks) to design and build scalable data pipelines in a cloud environment, with emphasis on performance and reliability. You will work with PySpark, Python, SQL, Delta Lake, and Databricks Notebooks, collaborating with stakeholders, shaping data models, and implementing CI/CD with Git and Azure DevOps.

The role requires a strong background in data warehousing, governance, and security, and offers the chance to impact financial

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field.
• Proven hands-on experience with Azure Databricks and large-scale data processing.
• Strong proficiency in Python, PySpark, and SQL.
• Experience with cloud platforms, preferably Microsoft Azure (AWS/GCP also considered).
• Knowledge of Data Modeling and Data Warehousing.
• Experience with version control systems and CI/CD tools such as Git and Azure DevOps.
• Understanding of data quality, governance, and best practices for enterprise data platforms.
• Ability to work independently while collaborating within Agile teams.

Responsabilités

• Design, develop, and optimize scalable data pipelines using Azure Databricks, PySpark, and SQL.
• Build and maintain data solutions leveraging Spark, Delta Lake, Databricks Notebooks, and Workflows.
• Develop robust ETL/ELT processes to ingest, transform, and deliver high-quality data.
• Collaborate with stakeholders to understand data requirements and implement solutions.
• Participate in data modeling, warehousing, and architecture discussions.
• Implement CI/CD practices for data projects using Git and Azure DevOps.
• Ensure data quality, governance, security, and compliance standards are met.
• Support performance tuning and troubleshooting of data workloads.

Connaissances

PySpark

Python

SQL

Formation

Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field

Outils

Azure Databricks

Git

Azure DevOps

Description du poste

We are HCLTech, one of the fastest-growing large tech companies in the world and home to 225,000+ people across 60 countries, supercharging progress through industry-leading capabilities centered around Digital, Engineering and Cloud. The driving force behind that work, our people, are diverse, creative, and passionate, raising the bar for excellence on a regular basis. We, in turn, work hard to bring out the best in them as we strive to help them find their spark and become the best version of themselves that they can be. If all this sounds like an environment you’ll thrive in, then you’re in the right place. Join us on our journey in advancing the technological world through innovation and creativity.

We are looking for an experienced Data Engineer (Databricks) to join our team and drive the development of scalable, high-performance data solutions in a cloud environment.

Key Responsibilities

• Design, develop, and optimize scalable data pipelines using Azure Databricks, PySpark, and SQL.
• Build and maintain data solutions leveraging Spark, Delta Lake, Databricks Notebooks, and Workflows.
• Develop robust ETL/ELT processes to ingest, transform, and deliver high-quality data.
• Collaborate with business and technical stakeholders to understand data requirements and implement efficient solutions.
• Participate in data modeling, warehousing, and architecture discussions.
• Implement CI/CD practices for data engineering projects using Git and Azure DevOps.
• Ensure data quality, governance, security, and compliance standards are met.
• Support performance tuning, troubleshooting, and optimization of data workloads.

Mandatory Skills

• PySpark
• Python
• SQL

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field.
• Proven hands-on experience with Azure Databricks and large-scale data processing.
• Strong proficiency in Python, PySpark, and SQL.
• Experience working with cloud platforms, preferably Microsoft Azure (AWS/GCP experience also considered).
• Strong understanding of:
• Data Modeling
• Data Warehousing
• Experience with version control systems and CI/CD tools such as Git and Azure DevOps.
• Knowledge of data quality, governance, and best practices for enterprise data platforms.
• Ability to work independently while effectively collaborating within Agile teams.

Preferred Skills

• Experience with financial services, asset management, or investment data
• Understanding of pricing, positions, transactions, and market data domains
• Exposure to data governance and metadata management frameworks
• Experience in performance optimization of Spark workloads

_________________________________________________________________________

We promote equal opportunities for all employees, regardless of their cultural and social background, gender, disability, age, religion, beliefs, and sexual identity. We give priority consideration to severely disabled applicants and those of equal status in the case of equal suitability.