Mid Level Data Engineer - KT

DCI CONSULTANTS PRIVATE LIMITED; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

DCI Consultants Private Limited in Singapore is seeking a Data Engineer to design, build, and maintain data infrastructure supporting statistical production processes and to help migrate legacy systems. You will contribute to modernising pipelines while preserving business logic and ensuring outputs align with current production results.

You will develop ETL pipelines, model data, implement CI/CD, and collaborate with statisticians and business stakeholders to deliver reliable data services and

Qualifications

• 4–7 years of professional data engineering experience.
• Proficient in Python and SQL with data transformations.
• Experience with ETL/data pipelines and data warehousing.
• Experience with Git and CI/CD pipelines.
• Strong Linux/Unix knowledge.

Responsibilities

• Design, develop and maintain ETL processes and data pipelines from multiple sources.
• Develop modular production-quality code for statistical processes and optimise legacy systems.
• Design and maintain database schemas, data models and storage structures.
• Build reusable data engineering components and utilities for production workflows.
• Implement and maintain CI/CD pipelines for data workflows and manage environments.

Skills

Python

SQL

Linux/Unix

Shell scripting

Data governance

Git

Polars

Pandas

GitLab

Cloud platforms

Agile methodologies

Communication

Problem-solving

Tools

Git

Polars

Pandas

GitLab

CI/CD

Airflow

Job description

Role Overview

The Data Engineer works closely with statisticians, analytical teams, and business stakeholders to design, build, and maintain the data infrastructure needed to support rigorous statistical production processes. The role centres on building reliable and maintainable data pipelines, validation and frameworks that support statistical production processes.

You will contribute to the migration and modernisation of legacy processes, ensuring that existing business logic is preserved and that outputs are accurately reconciled against current production results. You will also contribute to the engineering practices required to operate the platform reliably in production.

Key Responsibilities

• Design, develop and maintain efficient and reliable ETL processes and data pipelines for the ingestion, transformation, and integration of data from multiple source agencies.
• Develop and maintain modular, production-quality code for statistical production processes, including optimising legacy processes while preserving business logic and reconciling outputs against existing results.
• Design and maintain database schemas, data models and storage structures to support efficient ingestion, validation, transformation and retrieval of large datasets.
• Build reusable data engineering components such as common libraries, configuration frameworks, logging standards, and utilities to streamline production workflows.
• Implement and maintain CI/CD pipelines for data engineering workflows, including automated testing and deployment, and establish workflows to safely isolate and synchronise Development, UAT, and Production environments.
• Design and configure complex pipeline workflows and schedules using enterprise tools, including dependencies, failure handling, recovery and operational monitoring.
• Establish version control structures using GitLab, set up code review frameworks, and produce technical documentation.
• Monitor and troubleshoot pipelines, perform root-case analysis and continuously improve operational reliability and system performance for large-scale datasets.
• Collaborate with technical team, business users and other stakeholders to translate business and statistical processing requirements into robust technical implementations.

Required Skills & Experience

• 4-7 years of professional experience in data engineering, data platform development, or a related software engineering role.
• Strong proficiency in Python and SQL, including experience in complex data transformations, analysis, and quality monitoring.
• Hands-on experience with dataframe processing libraries (e.g. Polars, pandas) for high-performance execution and memory-efficient processing of large datasets
• Experience with database management, data warehousing and developing ETL processes for data ingestion and integration
• Experience with version control using Git, including branching strategies, merge requests, code review, and CI/CD pipelines for automated testing and deployment
• Working knowledge of Linux/Unix environments, shell scripting and command-line tools for data processing
• Familiar with data governance policies, access control and security best practices.
• Understanding of software engineering practices such as modular design, configuration management, logging, exception handling, testing, and technical documentation.
• Experience with cloud technologies and orchestration/CI-CD tooling is an advantage
• Excellent problem-solving, analytical, and communication skills, with the ability to work independently in an agile team environment

Working Location : Central

**We regret to inform that only shortlisted candidates will be notified. Personal data collected will be used for recruitment purposes**