Senior Data Engineer – Data Platform & Modernisation

NEXBRIDGE RECRUITMENT PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Nexbridge Recruitment Pte Ltd in Singapore seeks an experienced Data Engineer to architect and build scalable ETL pipelines, integrating data from multiple sources.

You will develop production-grade Python/SQL code, design robust data models, implement CI/CD for deployment, and mentor junior engineers while ensuring reliability and performance of large-scale datasets.

Qualifications

• 5 years of professional experience in data engineering, data platform development or a related software engineering role, with experience designing and implementing production-grade data solutions at scale.
• Strong proficiency in Python and SQL, with experience in complex data transformations, analysis and data quality monitoring.
• Strong experience in batch processing, ETL, data integration and data pipeline architecture.
• Strong working knowledge of Linux/Unix environments, shell scripting and command-line tools for data processing.
• Strong understanding and practical application of software engineering practices, including modular design, configuration management, logging, exception handling, testing and technical documentation.
• Experience defining or applying engineering standards for production systems.
• Experience designing technical solutions and making engineering trade-off decisions involving performance, maintainability, scalability and operational reliability.
• Experience leading or providing technical guidance for data pipeline development, system migration or modernisation initiatives.
• Experience conducting code reviews and mentoring or providing technical guidance to other engineers.
• Experience with version control using Git, including branching strategies, merge requests, code reviews and CI/CD pipelines for automated testing and deployment.
• Experience with cloud platforms and orchestration or CI/CD tools is preferred.
• Strong problem-solving, analytical and communication skills, with the ability to work independently and provide technical leadership when resolving complex engineering problems.

Responsibilities

• Lead the architecture, design and implementation of ETL processes, workflows and data pipelines for the ingestion, transformation and integration of data from multiple source agencies.
• Build and maintain efficient and reliable large-scale batch and real-time data pipelines, including dependency management, failure handling, recovery and operational monitoring.
• Develop and maintain modular, production-quality code for statistical production processes, including the modernisation of legacy processes while preserving existing business logic and reconciling outputs against current production results.
• Review complex business logic and provide technical direction for data migration and modernisation initiatives.
• Lead database and data model design, including schema design, storage patterns and performance optimisation for large-scale statistical datasets.
• Design and drive the adoption of reusable data engineering components, including common libraries, configuration frameworks, logging standards and utilities to streamline production workflows and promote consistent engineering practices.
• Design and implement CI/CD pipelines for data engineering workflows to automate testing and deployment, while establishing workflows across Development, UAT and Production environments.
• Lead the design and configuration of complex pipeline workflows and schedules using enterprise tools, including dependency management, failure handling, recovery and operational monitoring.
• Establish version control structures using GitLab, including branching strategies, merge requests and code review frameworks, while maintaining technical architecture and implementation documentation.
• Provide technical reviews for solution designs, merge requests and implementation approaches, and promote appropriate engineering standards across the development team.
• Troubleshoot production incidents, monitor pipeline health and continuously optimise system performance for large-scale datasets.
• Lead investigations into complex production issues, identify root causes and implement preventive measures and performance improvements.
• Provide technical guidance and mentoring to data engineers on coding practices, solution design, troubleshooting and implementation decisions.
• Collaborate with technical teams, business users and stakeholders to translate complex business and statistical processing requirements into robust technical solutions.
• Identify technical risks, dependencies and improvement opportunities, and recommend appropriate design and implementation approaches.

Skills

Python

SQL

ETL

Data pipelines

CI/CD

Linux/Unix

Code reviews

Production systems

Troubleshooting

Documentation

Tools

Git

GitLab

CI/CD tools

Cloud platforms

Docker

Job description

Job Responsibilities

• Lead the architecture, design and implementation of ETL processes, workflows and data pipelines for the ingestion, transformation and integration of data from multiple source agencies.
• Build and maintain efficient and reliable large-scale batch and real-time data pipelines, including dependency management, failure handling, recovery and operational monitoring.
• Develop and maintain modular, production-quality code for statistical production processes, including the modernisation of legacy processes while preserving existing business logic and reconciling outputs against current production results.
• Review complex business logic and provide technical direction for data migration and modernisation initiatives.
• Lead database and data model design, including schema design, storage patterns and performance optimisation for large-scale statistical datasets.
• Design and drive the adoption of reusable data engineering components, including common libraries, configuration frameworks, logging standards and utilities to streamline production workflows and promote consistent engineering practices.
• Design and implement CI/CD pipelines for data engineering workflows to automate testing and deployment, while establishing workflows across Development, UAT and Production environments.
• Lead the design and configuration of complex pipeline workflows and schedules using enterprise tools, including dependency management, failure handling, recovery and operational monitoring.
• Establish version control structures using GitLab, including branching strategies, merge requests and code review frameworks, while maintaining technical architecture and implementation documentation.
• Provide technical reviews for solution designs, merge requests and implementation approaches, and promote appropriate engineering standards across the development team.
• Troubleshoot production incidents, monitor pipeline health and continuously optimise system performance for large-scale datasets.
• Lead investigations into complex production issues, identify root causes and implement preventive measures and performance improvements.
• Provide technical guidance and mentoring to data engineers on coding practices, solution design, troubleshooting and implementation decisions.
• Collaborate with technical teams, business users and stakeholders to translate complex business and statistical processing requirements into robust technical solutions.
• Identify technical risks, dependencies and improvement opportunities, and recommend appropriate design and implementation approaches.

Job Requirements

• 5 years of professional experience in data engineering, data platform development or a related software engineering role, with experience designing and implementing production-grade data solutions at scale.
• Strong proficiency in Python and SQL, with experience in complex data transformations, analysis and data quality monitoring.
• Strong experience in batch processing, ETL, data integration and data pipeline architecture.
• Strong working knowledge of Linux/Unix environments, shell scripting and command-line tools for data processing.
• Strong understanding and practical application of software engineering practices, including modular design, configuration management, logging, exception handling, testing and technical documentation.
• Experience defining or applying engineering standards for production systems.
• Experience designing technical solutions and making engineering trade-off decisions involving performance, maintainability, scalability and operational reliability.
• Experience leading or providing technical guidance for data pipeline development, system migration or modernisation initiatives.
• Experience conducting code reviews and mentoring or providing technical guidance to other engineers.
• Experience with version control using Git, including branching strategies, merge requests, code reviews and CI/CD pipelines for automated testing and deployment.
• Experience with cloud platforms and orchestration or CI/CD tools is preferred.
• Strong problem-solving, analytical and communication skills, with the ability to work independently and provide technical leadership when resolving complex engineering problems.

By submitting your personal data and CV, you consent toNexbridge Recruitment Pte Ltdcollecting, using and disclosing your personal data for recruitment purposes.