Python Pyspark Developer

Unison Group; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

Unison Group seeks a data engineering professional in Singapore to design, build, and maintain PySpark-based ETL pipelines for large-scale data processing in a banking environment.

You will leverage Python, SQL and cloud platforms (AWS/Azure/Databricks) to transform data, ensure quality, and optimize performance in an Agile team with strong stakeholder collaboration.

Qualifications

• Strong hands-on experience in Python and PySpark for large-scale data processing.
• Design, develop and maintain ETL/data pipelines using PySpark, Python and SQL.
• Experience with Big Data technologies, data transformation and batch processing.
• Strong knowledge of SQL, data modelling, performance tuning and optimization.
• Experience with cloud/data platforms such as AWS, Azure, Databricks or equivalent.
• Good understanding of data quality, reconciliation, validation and error handling.
• Ability to analyse business requirements and translate them into scalable technical solutions.
• Experience working in Agile/Scrum environments with strong communication and stakeholder-management skills.
• Candidates should be available to work in a Singapore-based banking environment.

Responsibilities

• Design, develop and maintain ETL/data pipelines using PySpark, Python and SQL.
• Build and optimize large-scale data processing jobs in Python and PySpark.
• Work with cloud platforms (AWS/Azure/Databricks) to deploy data solutions.
• Ensure data quality, validation, reconciliation and error handling across pipelines.
• Collaborate with stakeholders to translate business needs into scalable data architectures.
• Participate in Agile/Scrum ceremonies and deliver iterative improvements.

Skills

Python

PySpark

ETL pipelines

Big Data

SQL

Data modelling

Performance tuning

Agile/Scrum

Stakeholder management

Data quality

Cloud platforms

Databricks

AWS

Azure

Tools

Databricks

AWS

Azure

Job description

Job Description:

• Strong hands-on experience in Python and PySpark development for large-scale data processing.
• Design, develop and maintain ETL/data pipelines using PySpark, Python and SQL.
• Experience working with Big Data technologies, data transformation and batch processing.
• Strong knowledge of SQL, data modelling, performance tuning and optimization.
• Experience with cloud/data platforms such as AWS, Azure, Databricks or equivalent is preferred.
• Good understanding of data quality, reconciliation, validation and error handling.
• Ability to analyse business requirements and translate them into scalable technical solutions.
• Experience working in Agile/Scrum environments with strong communication and stakeholder-management skills.
• Candidates should be available to work in a Singapore-based banking environment.