Software Engineer

RISKDATA CONSULTING PTE. LTD.; Singapore; On-site; SGD 90,000 - 150,000; Full time

Job summary

RISKDATA CONSULTING PTE. LTD. is seeking a data engineer with hands-on Python and PySpark experience to support large-scale data processing in a Singapore banking environment.

You will design, develop and maintain ETL pipelines using PySpark, Python and SQL, focusing on data quality, transformation and performance.

Experience with AWS, Azure or Databricks, Agile/Scrum practices and strong stakeholder communication is preferred.

Qualifications

• Proficient in Python and PySpark for large-scale data processing.
• Strong SQL skills, data modelling and performance tuning.
• Experience with data quality, validation and error handling.
• Translate business requirements into scalable data solutions.
• Agile/Scrum experience with strong stakeholder communication.

Responsibilities

• Design, develop and maintain ETL/data pipelines using PySpark, Python and SQL.
• Ensure data quality and error handling in production pipelines.
• Collaborate with stakeholders in a Singapore-based banking environment.
• Optimize performance and scalability of data processing jobs.

Skills

Python

PySpark

SQL

Data modelling

Data quality

Agile/Scrum

Stakeholder management

Tools

Databricks

AWS

Azure

Job description

Strong hands-on experience in Python and PySpark development for large-scale data processing.

Design, develop and maintain ETL/data pipelines using PySpark, Python and SQL.

Experience working with Big Data technologies, data transformation and batch processing.

Strong knowledge of SQL, data modelling, performance tuning and optimization.

Experience with cloud/data platforms such as AWS, Azure, Databricks or equivalent is preferred.

Good understanding of data quality, reconciliation, validation and error handling.

Ability to analyse business requirements and translate them into scalable technical solutions.

Experience working in Agile/Scrum environments with strong communication and stakeholder-management skills.

Candidates should be available to work in a Singapore-based banking environment.