Data Engineer: Scalable Pipelines & Production Reliability

HCL Technologies Limited; Singapore; On-site; SGD 90,000 - 140,000; Full time

Job summary

HCL Technologies Limited seeks a Data Engineer to join its Singapore team, collaborating with statisticians and analysts to design, build, and maintain scalable data pipelines that support statistical production processes.

You will migrate and modernise legacy processes while preserving business logic and ensuring outputs align with current production results; contribute to production-grade engineering practices for reliability and performance.

Qualifications

• Proficient in Python and SQL for data transformations, validation and monitoring.
• Experience building reliable ETL/data pipelines and workflow orchestration.
• Familiar with dataframe libraries (Polars, Pandas) for large datasets.
• Knowledge of Linux/Unix, shell scripting and version control workflows (Git).
• Understanding of data governance and security best practices.
• Strong communication and ability to work in agile teams.

Responsibilities

• Design, develop and maintain ETL processes and data pipelines from multiple sources.
• Build modular production code for statistical processes and optimise legacy pipelines.
• Design and maintain database schemas and data models for large datasets.
• Create reusable data engineering components and utilities for production workflows.
• Implement and maintain CI/CD pipelines with automated testing and deployment.
• Configure complex pipeline workflows with dependencies, failure handling and monitoring.
• Establish GitLab version control, code reviews, and technical documentation.
• Monitor pipelines, troubleshoot issues and improve reliability and performance.
• Collaborate with business users to translate requirements into robust tech solutions.

Skills

Python

SQL

Data pipelines

Polars/Pandas

CI/CD

Git/GitLab

Linux/Unix

Data governance

Communication

Tools

Git

GitLab

CI/CD tooling

Job description

HCL Technologies Limited seeks a Data Engineer to join its Singapore team, collaborating with statisticians and analysts to design, build, and maintain scalable data pipelines that support statistical production processes.

You will migrate and modernise legacy processes while preserving business logic and ensuring outputs align with current production results; contribute to production-grade engineering practices for reliability and performance.