Data Engineer

HCLTech

Amsterdam

On-site

EUR 50,000 - 90,000

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

An established industry player is seeking a highly skilled Python Data Engineer to join their dynamic team in Amsterdam. This exciting role focuses on the Finance and Risk domain, where you will leverage your expertise in Python, PySpark, and Databricks to analyze business requirements and design robust technical solutions. You will oversee the end-to-end development process, ensuring seamless migration and optimization of activities, while collaborating with a diverse team of professionals. If you are passionate about data engineering and want to make a significant impact in a fast-paced environment, this opportunity is perfect for you.

Qualifications

Extensive experience in analyzing business requirements and designing technical solutions.
Strong proficiency in Python, PySpark, and Databricks for data pipeline development.

Responsibilities

Analyze business documents related to Finance and Risk domain for technical requirements.
Design and maintain scalable data pipelines and workflows using Python and PySpark.

Skills

Python

PySpark

Data Analysis

Analytical Models

Data Quality Assessment

Tools

Databricks

Azure Pipeline

Job description

Duration: Permanent Position

Work Location: Amsterdam, Netherlands

Job Description:

Seeking a highly skilled Python Data Engineer with expertise in Finance and Risk domain to join our dynamic team. The ideal candidate will have extensive experience in analyzing business requirements, designing technical solutions, implementing complex business logic, ensuring seamless transition and optimization of activities within the migration process. The position requires strong proficiency in Python, PySpark, and Databricks, experience with analytical models, and a solid understanding of data structures and quality. The candidate will oversee end-to-end development, from requirement analysis to deployment, while ensuring robust testing and validation processes.

Key Responsibilities:

Requirement analysis: Analyse and interpret business documents related to Finance and Risk domain, especially corporate credit risk analytical models.
Collaborate with stakeholders to translate business needs into clear technical requirements.
Data analysis and understanding: Analyse and interpret data, understand data structures and concepts, assess data quality, identify inconsistencies, and resolve issues as needed.
Design, build, and maintain: Scalable data pipelines, workflows using Python, PySpark, and Databricks. Enhance performance by leveraging the capabilities of Python and PySpark.
Testing and Validation: Conduct thorough testing and validation of implementation to ensure they meet performance and functionality standards, ensuring comprehensive test coverage.
Perform review, debugging, troubleshooting, and issue resolution.
Deployment and Maintenance: Deploy solutions using Azure Pipeline and monitor and maintain deployed solutions.
Code and Data Conversion: Convert SAS code of analytical assets to Python or PySpark, ensuring accuracy and efficiency of the new programs.
Migration Planning: Contribute to developing a comprehensive migration plan, including timelines, resource allocation, and risk management.
Evaluate all work to ensure compliance with internal Privacy and Security Policies and Procedures and migration project practices.
Meet timelines and milestones by monitoring deliverables; identify, report, and help solve potential risks and issues.
Documentation: Maintain detailed documentation of migration processes, including code changes, testing procedures, and performance metrics.
Collaboration: Work closely with business analysts, modelling team, SAS developers, data scientists, data engineers, and other stakeholders to ensure successful migration and integration of programs.
Training and Support: Provide training and support to team members on the new Databricks, Python/PySpark-based programs.