Data Engineer

nSearch Global Pte Ltd.; Singapore; Sur place; SGD 90 000 - 130 000; Plein temps

Résumé du poste

nSearch Global Pte Ltd. in Singapore is seeking a data engineer to design, develop and maintain robust ETL pipelines for ingestion, transformation and integration of data from multiple sources. You will build scalable components and dashboards for production use.

Strong Python and SQL skills, experience with dataframe libraries (Polars, pandas), GitLab CI/CD, and Linux shell scripting are required. The role offers collaboration across teams to improve reliability and performance on large

Qualifications

• Degree with 4-7 years of professional experience in data engineering or related software role.
• Strong Python and SQL with complex data transformations and quality monitoring.
• Hands-on experience with dataframe processing libraries (e.g. Polars, pandas) for high-performance execution and memory-efficient processing of large datasets.
• Experience with database management, data warehousing and ETL processes for data ingestion and integration.
• Experience with Git, branching strategies, merge requests, code review, and CI/CD pipelines for automated testing and deployment.
• Good knowledge of Linux/Unix environments, shell scripting and CLI tools for data processing.
• Experience with cloud platforms and orchestration/CI‑CD tooling is preferred.

Responsabilités

• Design, develop and maintain ETL processes and data pipelines for ingestion, transformation and integration.
• Develop modular, production-quality code for statistical production processes and optimise legacy processes.
• Design and maintain database schemas, data models and storage structures for large datasets.
• Build reusable data engineering components, libraries, configuration frameworks, logging standards and utilities.
• Implement and maintain CI/CD pipelines for data engineering workflows with automated testing and deployment.
• Establish version control with GitLab, set up code review frameworks, and produce technical documentation.
• Monitor and troubleshoot pipelines, perform root-cause analysis and improve reliability and performance for large-scale datasets.

Connaissances

Python

SQL

Data transformations

Data quality monitoring

Shell scripting

Linux/Unix

Formation

Bachelor's degree in Computer Science or related field

Outils

Git

GitLab

Polars

pandas

CI/CD

Description du poste

Our client, one of Asia-Pacific’s leading organizations is looking for:

Responsibilities:

• Design, develop and maintain efficient and reliable ETL processes and data pipelines for the ingestion, transformation, and integration of data from multiple source agencies
• Develop and maintain modular, production-quality code for statistical production processes, including optimising legacy processes while preserving business logic and reconciling outputs against existing results
• Design and maintain database schemas, data models and storage structures to support efficient ingestion, validation, transformation and retrieval of large datasets
• Build reusable data engineering components such as common libraries, configuration frameworks, logging standards, and utilities to streamline production workflows
• Implement and maintain CI/CD pipelines for data engineering workflows, including automated testing and deployment
• Establish version control structures using GitLab, set up code review frameworks, and produce technical documentation
• Monitor and troubleshoot pipelines, perform root‑cause analysis and continuously improve operational reliability and system performance for large-scale datasets

Requirements:

• Degree with minimum 4-7 years of professional experience in data engineering, data platform development, or a related software engineering role
• Strong proficiency in Python and SQL, including experience in complex data transformations, analysis, and quality monitoring.
• Hands‑on experience with dataframe processing libraries (e.g. Polars, pandas) for high‑performance execution and memory‑efficient processing of large datasets
• Experience with database management, data warehousing and developing ETL processes for data ingestion and integration
• Experience with version control using Git, including branching strategies, merge requests, code review, and CI/CD pipelines for automated testing and deployment
• Good knowledge of Linux/Unix environments, shell scripting and command-line tools for data processing
• Experience with version control using Git, including branching strategies, merge requests, code review, and CI/CD pipelines for automated testing and deployment
• Experience with cloud platforms and orchestration/CI‑CD tooling is preferred

Only shortlisted candidates will be notified.