Data Engineer (SQL / Python /PySpark)

OPTIMUM SOLUTIONS (SINGAPORE) PTE LTD; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

OPTIMUM SOLUTIONS (SINGAPORE) PTE LTD is seeking a hands-on Data Engineer to own and optimise scalable ETL pipelines, historical data architectures, and analytical datasets with a focus on data quality and reconciliation across the enterprise. You will work on batch and streaming pipelines, data lakes, and EDW transformations, ensuring secure data platforms and robust monitoring.

The role requires 6+ years of data engineering experience in banking data operations and a strong Spark/PySpark, SQL,

Qualifications

• 6+ years hands-on data engineering experience with independent ownership of development and testing deliverables.
• Advanced proficiency in SQL, Python, Spark/PySpark, Teradata, and Hadoop/Cloudera ecosystems.
• Experience with Informatica/Talend, Hive, Impala, Kafka, Airflow, APIs, and Git-based version control.
• Strong background in enterprise data warehousing, data lakes, and banking data operations (Payments, Risk, Finance, or AML preferred).
• Independent ownership of deliverables, analytical mindset, and commitment to secure, high-quality engineering standards.

Responsibilities

• Develop and maintain robust batch, streaming, CDC, and API-based data pipelines, historical snapshots, and SCD frameworks.
• Build transformations for EDW, data lakes, and semantic layers, including specialised payment harmonisation datasets.
• Tune SQL queries and Spark jobs, implementing stringent reconciliation controls, error handling, and operational monitoring.
• Support SIT, UAT, defect resolution, release execution, and production verification in close collaboration with architecture and BI teams.

Skills

SQL

Python

Spark/PySpark

Teradata

Hadoop/Cloudera

Banking data operations

Independent ownership

Data quality

Tools

Informatica

Talend

Hive

Impala

Kafka

Airflow

APIs

Git

Job description

About the Role

We are looking for a hands‑on Data Engineer to drive the core mechanics of our enterprise data platforms. In this role, you will independently build, optimise, and secure scalable ETL pipelines, historical data architectures, and analytical datasets, ensuring rigorous data quality and reconciliation across our ecosystem.

Responsibilities

• Pipeline Engineering: Develop and maintain robust batch, streaming, CDC, and API-based data pipelines, historical snapshots, and SCD frameworks.
• Data Transformation: Build transformations for EDW, data lakes, and semantic layers, including specialised payment harmonisation datasets.
• Performance & Quality: Tune SQL queries and Spark jobs, implementing stringent reconciliation controls, error handling, and operational monitoring.
• SDLC Delivery: Support SIT, UAT, defect resolution, release execution, and production verification in close collaboration with architecture and BI teams.

Requirements:

• Experience: Minimum 6 years of hands‑on data engineering experience, with independent ownership of development and testing deliverables.
• Technical Stack: Advanced proficiency in SQL, Python, Spark/PySpark, Teradata, and Hadoop/Cloudera ecosystems.
• Tools & Integration: Experience with Informatica/Talend, Hive, Impala, Kafka, Airflow, APIs, and Git-based version control.
• Domain Exposure: Strong background in enterprise data warehousing, data lakes, and banking data operations (Payments, Risk, Finance, or AML preferred).
• Core Competencies: Independent ownership of deliverables, analytical mindset, and a commitment to secure, high-quality engineering standards.