Python Pyspark Developer

Hong Kong Unison Limited; Singapore; On-site; SGD 90,000 - 150,000; Full time

Job summary

Hong Kong Unison Limited in Singapore seeks a data engineer to design, build and maintain ETL pipelines using PySpark, Python and SQL. You will work with Big Data technologies, ensure data quality, and optimize batch processes within a banking environment.

The role requires strong coding skills, data modelling and experience with cloud platforms (AWS/Azure/Databricks). You will participate in Agile/Scrum teams and collaborate with stakeholders across functions.

Qualifications

• Strong hands-on Python and PySpark development for large-scale data processing.
• Experience designing and maintaining ETL/data pipelines with PySpark, Python and SQL.
• Knowledge of Big Data technologies, data transformation and batch processing.
• Strong SQL data modelling and performance tuning skills.
• Experience with cloud platforms (AWS/Azure/Databricks) is preferred.
• Understanding of data quality, reconciliation, validation and error handling.
• Ability to translate business requirements into scalable solutions.
• Experience in Agile/Scrum with stakeholder management.

Responsibilities

• Design, develop and maintain ETL/data pipelines using PySpark, Python and SQL.
• Analyse business requirements and translate them into scalable technical solutions.
• Collaborate with stakeholders in a Singapore-based banking environment.

Skills

Python

PySpark

SQL

Data modelling

Performance tuning

Big Data

Agile/Scrum

Stakeholder management

Data quality

Tools

Databricks

AWS

Azure

Job description

• Strong hands-on experience in Python and PySpark development for large-scale data processing.
• Design, develop and maintain ETL/data pipelines using PySpark, Python and SQL.
• Experience working with Big Data technologies, data transformation and batch processing.
• Strong knowledge of SQL, data modelling, performance tuning and optimization.
• Experience with cloud/data platforms such as AWS, Azure, Databricks or equivalent is preferred.
• Good understanding of data quality, reconciliation, validation and error handling.
• Ability to analyse business requirements and translate them into scalable technical solutions.
• Experience working in Agile/Scrum environments with strong communication and stakeholder-management skills.
• Candidates should be available to work in a Singapore-based banking environment.

Researching careers? Find all the information and tips you need on career advice.